-- Logs begin at Fri 2022-09-02 08:26:23 UTC, end at Fri 2022-09-02 09:29:40 UTC. -- Sep 02 08:54:43.997572 dv-cinder-855359-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 02 08:54:44.004879 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 02 08:54:44.009931 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 2 08:54:44 2022] *** Sep 02 08:54:44.009931 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 02 08:54:44.009931 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 02 08:54:44.009931 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: nodename: dv-cinder-855359-2-iscsi Sep 02 08:54:44.009931 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: machine: x86_64 Sep 02 08:54:44.009931 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: clock source: unix Sep 02 08:54:44.009931 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: pcre jit disabled Sep 02 08:54:44.009931 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: detected number of CPU cores: 4 Sep 02 08:54:44.009931 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: current working directory: / Sep 02 08:54:44.009931 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: detected binary path: /usr/bin/uwsgi-core Sep 02 08:54:44.009931 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: your processes number limit is 47680 Sep 02 08:54:44.009931 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: your memory page size is 4096 bytes Sep 02 08:54:44.009931 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: detected max file descriptor number: 2048 Sep 02 08:54:44.009931 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: lock engine: pthread robust mutexes Sep 02 08:54:44.009931 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: thunder lock: enabled Sep 02 08:54:44.009931 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 02 08:54:44.009931 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 02 08:54:44.048473 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: Python main interpreter initialized at 0x560677dea9c0 Sep 02 08:54:44.048473 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: python threads support enabled Sep 02 08:54:44.048473 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: your server socket listen backlog is limited to 100 connections Sep 02 08:54:44.048473 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: your mercy for graceful operations on workers is 90 seconds Sep 02 08:54:44.048851 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: mapped 403077 bytes (393 KB) for 2 cores Sep 02 08:54:44.048935 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: *** Operational MODE: preforking *** Sep 02 08:54:44.049162 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: *** uWSGI is running in multiple interpreter mode *** Sep 02 08:54:44.049162 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: spawned uWSGI master process (pid: 81374) Sep 02 08:54:44.049371 dv-cinder-855359-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 02 08:54:44.055115 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: spawned uWSGI worker 1 (pid: 81389, cores: 1) Sep 02 08:54:44.055115 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: spawned uWSGI worker 2 (pid: 81390, cores: 1) Sep 02 08:54:44.055115 dv-cinder-855359-2-iscsi devstack@keystone.service[81374]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 02 08:54:48.316689 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 02 08:54:48.316689 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: from cryptography.utils import int_from_bytes Sep 02 08:54:48.317502 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 02 08:54:48.317502 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: from cryptography.utils import int_from_bytes Sep 02 08:54:48.317997 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 02 08:54:48.317997 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: from cryptography.utils import int_from_bytes Sep 02 08:54:48.318467 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 02 08:54:48.318467 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: from cryptography.utils import int_from_bytes Sep 02 08:54:48.367514 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 02 08:54:48.367651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 02 08:54:48.367735 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 02 08:54:48.367823 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 02 08:54:48.367904 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] command line args: [] {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 02 08:54:48.367986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] command line args: [] {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 02 08:54:48.368147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 02 08:54:48.368228 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 02 08:54:48.368306 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ================================================================================ {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 02 08:54:48.368383 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ================================================================================ {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 02 08:54:48.368565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] admin_token = **** {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.368658 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] admin_token = **** {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.368803 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] config_dir = [] {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.368883 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] config_dir = [] {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.368989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.369070 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.369172 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] config_source = [] {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.369263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] config_source = [] {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.369343 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] debug = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.369420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] debug = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.369532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.369784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.369884 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.369946 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.370081 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.370159 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.370298 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.370388 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.370535 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.370623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.370789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] list_limit = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.370929 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] list_limit = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.371034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] log_config_append = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.371182 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] log_config_append = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.371290 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.371393 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.371498 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] log_dir = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.371594 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] log_dir = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.371694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] log_file = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.371809 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] log_file = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.371897 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.372016 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.372098 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.372176 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.372298 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.372387 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.372462 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.372588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.372674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.372748 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.372876 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.372955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.373078 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.373155 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.373276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.373365 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.373450 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.373572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.373655 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.373727 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.373842 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.373917 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.374039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.374116 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.374262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.374362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.374504 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.374584 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.374776 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.374865 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.375282 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.375395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.36/identity {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.375483 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] publish_errors = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.375557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] publish_errors = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.375678 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.375763 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.375905 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.375984 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.376081 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.376160 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.376328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.376486 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.376580 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.376661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.376775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] standard_threads = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.376856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] standard_threads = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.377003 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.377082 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.377242 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.377320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.377414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.377495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.377631 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] use_journal = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.377706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] use_journal = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.377781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] use_json = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.377851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] use_json = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.377926 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] use_stderr = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.378001 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] use_stderr = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.378082 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] use_syslog = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.378154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] use_syslog = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.378279 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.378364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 08:54:48.378531 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.378609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.378763 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.378841 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.379015 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.379109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.379277 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.379357 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.379522 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.379620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.379758 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.379835 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.380002 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.380078 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.380232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.380316 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.380476 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.380560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.380720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.380797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.380964 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.381043 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.381193 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.381271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.381436 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.381513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.381668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.381747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.381911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.381989 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.382155 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.382239 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.382388 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.382465 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.382637 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.382713 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.382890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.383038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.383142 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.383224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.383376 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.383461 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.383623 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.383714 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.383882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.383965 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.384130 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.384213 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.384375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.384465 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.384606 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.384683 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.384832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.384908 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.385061 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.385146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.385292 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.385375 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.385538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.385614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.385782 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.385870 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.386010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.386087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.386235 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.386313 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.386997 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.387139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.387267 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.387345 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.387503 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.387581 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.387747 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.387832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.387976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.388052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.388207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.388285 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.388435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.388520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.388661 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.388744 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.388888 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.388971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.389114 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.389265 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.389340 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.389409 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.389559 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.389636 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.389803 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.389887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.390028 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.390105 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.390261 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.390337 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.390485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.390569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.390717 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.390796 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.390949 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.391027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.391213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.391293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.391441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.391551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.391675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.391784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.391909 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.392020 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.392142 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.392244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.392372 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.392498 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.392584 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.392699 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.392806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.392884 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.393051 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.393128 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.393280 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.393359 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.393513 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.393587 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.393739 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.393829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.393994 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.394080 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.394227 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.394306 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.394453 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.394531 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.394733 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.394830 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.394912 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.394980 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.395194 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.395282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.395418 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.395492 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.395662 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.395749 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.395902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.395988 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.396134 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.396213 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.396366 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.396444 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.396599 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.396711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.396843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.396961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.397077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.397189 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.397309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.397400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.397553 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.397659 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.397787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.397871 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.398028 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.398104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.398264 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.398355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.398497 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.398580 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.398727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.398804 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.398958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.399035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.399210 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.399290 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.399468 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.399546 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.399704 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.399785 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.399948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.400029 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.400187 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.400263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.400419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.400497 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.400641 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.400716 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.400887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.400965 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.401110 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.401197 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.401342 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.401430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.401569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.401647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.401793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.401882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.402019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.402110 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.402848 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.402949 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.403030 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.403128 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.403292 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.403385 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.403527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.403608 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.403773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.403861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.404009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.404097 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.404260 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.404340 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.404516 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.404603 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.404754 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.404865 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.404984 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] auth.external = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.405075 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] auth.external = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.405213 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.405300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.405506 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.405624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.405737 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.405863 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.405950 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] auth.password = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.406096 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] auth.password = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.406185 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] auth.token = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.406329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] auth.token = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.406409 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.406570 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.406651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.406809 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.406889 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.407063 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.407174 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.407319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.407401 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.407545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.407622 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.407775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.407856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.408008 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] credential.caching = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.408090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.408243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] credential.caching = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.408329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.408492 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.408572 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.408734 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.408814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.408973 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.409053 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.409214 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.409297 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.409447 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.409529 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.409680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.409760 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.409928 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.410008 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.410160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.410241 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.410405 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.410484 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.410657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.410747 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.410913 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.410998 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.411196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.411280 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.411440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.411533 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.411692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.411774 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.411925 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] federation.caching = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.412006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.412160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.412248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] federation.caching = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.412394 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.412475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.412636 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.412717 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.412864 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.412954 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.413098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.413178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.413330 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.413409 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.413572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.413659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.413803 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.413883 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.414051 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.414130 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.414283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.414373 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.414528 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.414608 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.414763 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] identity.caching = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.414843 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.414996 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] identity.caching = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.415177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.415262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.415335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.415486 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.415568 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.415720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.415809 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.415953 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.416034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.416181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.416260 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.416416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.416506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.416651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.416731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.416898 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.416979 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.417126 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.417218 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.417355 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.417437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.417584 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.417713 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.417836 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.417927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.418083 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.418163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.418328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.418407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.418573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.418668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.418816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.418902 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.419060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.419207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.419330 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.419411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.419566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.419645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.419795 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.419875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.420024 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.420105 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.420250 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.420329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.420478 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.420556 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.420714 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.420796 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.420949 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.421028 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.421186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.421265 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.421412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.421493 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.421653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.421732 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.421893 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.421972 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.422123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.422205 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.422359 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.422438 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.422598 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.422677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.422825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.422908 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.423059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.423156 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.423305 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.423384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.423532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.423613 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.423762 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.423841 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.423996 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.424075 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.424236 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.424318 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.424473 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.424553 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.424709 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.424789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.424958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.425040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.425203 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.425283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.425432 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.425512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.425661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.425743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.425908 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.425989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.426143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.426223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.426375 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.426456 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.426607 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.426687 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.426835 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.user = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.426916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.427125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.user = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.427211 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.427332 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.427412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.427563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.427642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.427808 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.427891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.428052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.428132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.428294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.428373 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.428532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.428614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.428759 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.428839 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.428986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.429066 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.429217 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.429297 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.429447 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.429526 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.429673 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.429752 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.429900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.429981 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.430145 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.430227 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.430385 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.430464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.430627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.430708 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.430867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.430948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.431112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.431196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.431358 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.431441 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.431592 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.431673 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.431825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.431906 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.432058 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.432139 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.432287 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.432367 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.432519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.432598 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.432760 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.432841 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.432994 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.433074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.433228 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.433308 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.433482 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.433563 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.433723 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.433803 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.433955 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.434035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.434187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.434269 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.434420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.434499 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.434655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.434744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.434890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.434970 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.435146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.435229 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.435378 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.435469 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.435614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] resource.caching = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.435694 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.435853 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] resource.caching = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.435931 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.436095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.436182 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.436329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.436410 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.436564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.436643 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.436809 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.436896 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.437040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.437126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.437282 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.437362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.437515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.437719 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.437801 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.437873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.437971 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] role.caching = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.438051 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.438204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] role.caching = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.438291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] role.driver = sql {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.438435 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.438515 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] role.driver = sql {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.438667 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.438753 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.438910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.438999 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.439176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.439258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.439691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.439777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.439932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.440026 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.440174 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.440263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.440416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.440497 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.440659 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.440749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.440885 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.440964 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.441125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.441204 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.441365 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.441453 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.441609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.441688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.441849 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.441928 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.442092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.442233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.442466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.442545 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.442702 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.442781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.442945 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.443027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.443212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.443294 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.443457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.443537 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.443688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.443771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.443922 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.444002 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.444174 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.444253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.444410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.444492 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.444639 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.444720 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.444868 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.444948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.445098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.445179 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.445329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.445409 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.445581 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.445659 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.445823 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.445905 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.446059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.446139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.446293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.446372 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.446520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.446601 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.446746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.446825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] token.caching = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.446979 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.447057 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] token.caching = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.447240 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.447321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.447473 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.447552 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.447711 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.447790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.447958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.448039 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.448193 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.448273 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.448426 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.448508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.448669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.448749 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.448907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.449471 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.449653 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.449731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.449890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.449970 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.450121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.450200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.450353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.450431 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.450598 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.450678 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.450838 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.450925 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.451126 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.451213 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.451341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.451431 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.451602 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] database.connection = **** {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.451690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.451844 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.451925 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] database.connection = **** {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.452087 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.452166 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.452322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.452409 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.452557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.452637 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.452788 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.452867 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.453019 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.453107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.453250 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.453330 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.453481 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.453560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.453715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.453804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.453946 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.454025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.454178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.454258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.454408 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.454496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.454641 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.454721 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.454872 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.454950 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.455127 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.455211 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.455357 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.455437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.455588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.455667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.455818 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.455899 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.455973 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81389) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 02 08:54:48.456045 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 08:54:48.456205 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81390) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 02 08:54:48.485508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.485665 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.485748 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.485882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.485967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.486113 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.494185 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.494680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.522201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.522433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.522567 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.522756 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.575149 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.575505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.623117 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.623351 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.623925 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.624162 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.778465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.778734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.778872 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.779002 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.779175 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.779310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.779436 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.779574 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.779717 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.779891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.780021 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.780146 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.802507 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.802697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.802891 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.803053 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.803211 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.803337 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.803463 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.803600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.803737 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.803863 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.803988 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.804112 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.804236 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.804368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.815721 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.815868 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.829019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.829197 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.829357 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.829485 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.829625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.829758 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.829887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.830015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.830152 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.830281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.830405 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.830530 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.830664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81389) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.830799 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81390) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 08:54:48.846952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.847253 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 02 08:54:48.847385 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.847518 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 02 08:54:48.847658 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.847788 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 02 08:54:48.847916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.848044 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 02 08:54:48.848171 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.848313 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.848450 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.848579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.848754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.848885 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.849012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.849265 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.858369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.858627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.858755 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.858905 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.859030 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.859198 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.859327 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.859457 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.866187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.866423 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.869093 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.869364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.869450 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.869647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.869772 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.869911 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.870051 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.870181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.870309 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.870437 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.870563 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.870698 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.871342 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.871498 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.871626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.871755 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.871910 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.872038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.872166 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.872292 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.872418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.872675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.895568 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.895887 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.897009 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.897293 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.897425 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.897623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.901959 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.902268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.902394 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.902549 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.902688 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.902894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.908897 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.909187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.909311 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.909557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.913686 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.913977 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.914262 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.914546 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.914734 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.914872 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.915030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.915175 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.915338 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.915465 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.915613 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.915744 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.915872 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.916144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.916413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.916676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.932169 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.932447 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.932653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.932933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.936854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.937347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.940389 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.940686 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.940864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.941164 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.944985 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.945422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.948140 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.948616 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.951927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.952225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.952407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.952535 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.952711 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.952843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.953004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.953135 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.953290 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.953421 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.953576 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.953856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.969304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.969590 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.969842 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.969968 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.970116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.970253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.970383 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.970643 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.977041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.977589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.978565 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.979147 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.980166 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.980444 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.980732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.981009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.983100 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.983395 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.983676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.983807 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.983960 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.984110 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.984361 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.984491 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.984651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.984952 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.995713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.996512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.996709 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:48.996837 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.996998 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.997136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.997387 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.997516 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.997673 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.997800 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.997955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.998082 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.998231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:48.998524 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.013968 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:49.014593 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:49.016889 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:49.017531 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:49.019774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.020464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.021214 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.021899 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.022680 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:49.022971 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.023400 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.023530 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:49.023814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.024104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.029786 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.031290 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:49.031874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.033360 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:49.034145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.034539 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.034824 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.035132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.036374 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.036671 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.036957 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.037242 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.043486 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:49.043773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.044052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.045613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:49.045904 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.046185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.050939 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81389) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:49.051246 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.051527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.051817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.052089 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.052363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.052642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.052915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.053153 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81390) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 08:54:49.053280 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.053439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.053565 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.053705 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.053848 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.053981 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.054108 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.054242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.054367 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.054512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81389) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.054636 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.054783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.055088 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.055378 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.055656 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.055928 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.056191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.056457 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.056733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81390) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 08:54:49.089098 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x560677dea9c0 pid: 81389 (default app) Sep 02 08:54:49.091616 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x560677dea9c0 pid: 81390 (default app) Sep 02 08:54:49.092296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cb4f1-dc8e-4a02-bffd-5a90fe0230cf None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:54:49.092582 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cb4f1-dc8e-4a02-bffd-5a90fe0230cf None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:54:49.092852 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cb4f1-dc8e-4a02-bffd-5a90fe0230cf None None] PATH_INFO: `/v3` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:54:49.096938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1/1] 10.222.0.36 () {50 vars in 791 bytes} [Fri Sep 2 08:54:49 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:54:55.622084 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87156506-4460-4711-9119-6c05fccbc833 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:54:55.622606 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87156506-4460-4711-9119-6c05fccbc833 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:54:55.622606 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87156506-4460-4711-9119-6c05fccbc833 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:54:55.626806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1/2] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:54:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:54:55.632009 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e52eb-30ee-4bd2-91e3-e5beb906a72b None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:54:55.632543 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e52eb-30ee-4bd2-91e3-e5beb906a72b None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:54:55.632543 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e52eb-30ee-4bd2-91e3-e5beb906a72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:54:55.703303 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: INFO dbcounter [None req-ff0e52eb-30ee-4bd2-91e3-e5beb906a72b None None] Registered counter for database keystone Sep 02 08:54:55.722072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG oslo_db.sqlalchemy.engines [None req-ff0e52eb-30ee-4bd2-91e3-e5beb906a72b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81389) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 02 08:54:55.728280 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writer thread running {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 02 08:54:55.907014 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG passlib.handlers.bcrypt [None req-ff0e52eb-30ee-4bd2-91e3-e5beb906a72b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81389) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 02 08:54:55.907344 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG passlib.handlers.bcrypt [None req-ff0e52eb-30ee-4bd2-91e3-e5beb906a72b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81389) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 02 08:54:55.965859 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-ff0e52eb-30ee-4bd2-91e3-e5beb906a72b None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:54:55.967484 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-ff0e52eb-30ee-4bd2-91e3-e5beb906a72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:54:56.094259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 2/3] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:54:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 02 08:54:56.543001 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87156506-4460-4711-9119-6c05fccbc833 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:54:56.611860 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: INFO dbcounter [None req-87156506-4460-4711-9119-6c05fccbc833 None None] Registered counter for database keystone Sep 02 08:54:56.632947 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG oslo_db.sqlalchemy.engines [None req-87156506-4460-4711-9119-6c05fccbc833 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81390) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 02 08:54:56.639596 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writer thread running {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 02 08:54:56.759156 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87156506-4460-4711-9119-6c05fccbc833 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:54:56.764011 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3055de3-4979-48cf-9720-297ae8633c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:54:56.764641 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3055de3-4979-48cf-9720-297ae8633c7e admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:54:56.764922 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3055de3-4979-48cf-9720-297ae8633c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:54:56.765195 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3055de3-4979-48cf-9720-297ae8633c7e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:54:56.773312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3055de3-4979-48cf-9720-297ae8633c7e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:54:56.825414 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a3055de3-4979-48cf-9720-297ae8633c7e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 02 08:54:56.825756 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3055de3-4979-48cf-9720-297ae8633c7e admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:54:56.835096 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 2/4] 10.222.0.36 () {58 vars in 1324 bytes} [Fri Sep 2 08:54:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 02 08:54:56.841646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e52eb-30ee-4bd2-91e3-e5beb906a72b None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:54:56.856311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e52eb-30ee-4bd2-91e3-e5beb906a72b None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:54:56.860668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822a7f6-e387-4a63-9e1f-7fce29d5c182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:54:56.861745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822a7f6-e387-4a63-9e1f-7fce29d5c182 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:54:56.862097 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822a7f6-e387-4a63-9e1f-7fce29d5c182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:54:56.862338 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822a7f6-e387-4a63-9e1f-7fce29d5c182 admin admin] PATH_INFO: `/v3/projects/32923700004a4c11b8926f21bae6d460` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:54:56.863914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6822a7f6-e387-4a63-9e1f-7fce29d5c182 admin admin] RBAC: Authorizing `identity:get_project(project_id=32923700004a4c11b8926f21bae6d460)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:54:56.915515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6822a7f6-e387-4a63-9e1f-7fce29d5c182 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 02 08:54:56.915857 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6822a7f6-e387-4a63-9e1f-7fce29d5c182 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:54:56.920887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 3/5] 10.222.0.36 () {56 vars in 1207 bytes} [Fri Sep 2 08:54:56 2022] GET /identity/v3/projects/32923700004a4c11b8926f21bae6d460 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:54:58.803574 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61c053-876d-442d-8791-bbb6d8ba9bdf None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:54:58.804149 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61c053-876d-442d-8791-bbb6d8ba9bdf None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:54:58.804149 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61c053-876d-442d-8791-bbb6d8ba9bdf None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:54:58.808589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 3/6] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:54:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:54:58.814244 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0262e-5db2-4a0e-8260-bade2e49845c None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:54:58.814627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0262e-5db2-4a0e-8260-bade2e49845c None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:54:58.814801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae0262e-5db2-4a0e-8260-bade2e49845c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:54:58.835260 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-4ae0262e-5db2-4a0e-8260-bade2e49845c None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:54:58.836773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-4ae0262e-5db2-4a0e-8260-bade2e49845c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:54:58.845060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 4/7] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:54:58 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 02 08:54:59.117246 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61c053-876d-442d-8791-bbb6d8ba9bdf None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:54:59.129100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61c053-876d-442d-8791-bbb6d8ba9bdf None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:54:59.133185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2e450-48ef-43d5-a4da-4d9aeafa0456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:54:59.133777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2e450-48ef-43d5-a4da-4d9aeafa0456 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:54:59.134053 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2e450-48ef-43d5-a4da-4d9aeafa0456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:54:59.134320 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2e450-48ef-43d5-a4da-4d9aeafa0456 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:54:59.141734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2e450-48ef-43d5-a4da-4d9aeafa0456 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:54:59.143526 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2e450-48ef-43d5-a4da-4d9aeafa0456 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:54:59.152926 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 4/8] 10.222.0.36 () {58 vars in 1324 bytes} [Fri Sep 2 08:54:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 08:54:59.158189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0262e-5db2-4a0e-8260-bade2e49845c None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:54:59.167940 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae0262e-5db2-4a0e-8260-bade2e49845c None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:54:59.172257 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373245d-114a-4929-b584-2ecc4a6d25a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:54:59.172881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373245d-114a-4929-b584-2ecc4a6d25a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:54:59.173165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373245d-114a-4929-b584-2ecc4a6d25a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:54:59.173441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373245d-114a-4929-b584-2ecc4a6d25a8 admin admin] PATH_INFO: `/v3/users/95efeb6f9f5f497f9453dc5e9764070a` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:54:59.174739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373245d-114a-4929-b584-2ecc4a6d25a8 admin admin] RBAC: Authorizing `identity:get_user(user_id=95efeb6f9f5f497f9453dc5e9764070a)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:54:59.176599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5373245d-114a-4929-b584-2ecc4a6d25a8 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:54:59.178207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 5/9] 10.222.0.36 () {56 vars in 1198 bytes} [Fri Sep 2 08:54:59 2022] GET /identity/v3/users/95efeb6f9f5f497f9453dc5e9764070a => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:01.124296 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d30563-081b-4200-aecf-46d145789472 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:01.124833 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d30563-081b-4200-aecf-46d145789472 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:01.124833 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d30563-081b-4200-aecf-46d145789472 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:01.126153 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 5/10] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:01.131350 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecf46a-1bc5-4b53-9ab1-26d929de2a61 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:01.131764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecf46a-1bc5-4b53-9ab1-26d929de2a61 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:01.131897 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecf46a-1bc5-4b53-9ab1-26d929de2a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:01.149499 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-70ecf46a-1bc5-4b53-9ab1-26d929de2a61 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:01.151153 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-70ecf46a-1bc5-4b53-9ab1-26d929de2a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:01.178477 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373064a-4231-4cb0-b67c-8415d23274bb None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:01.178761 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373064a-4231-4cb0-b67c-8415d23274bb None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:01.179040 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373064a-4231-4cb0-b67c-8415d23274bb None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:01.180384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 6/11] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:55:01.185592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc899a6f-435f-48e7-8938-9fbaab8b01ca None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:01.185873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc899a6f-435f-48e7-8938-9fbaab8b01ca None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:01.186145 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc899a6f-435f-48e7-8938-9fbaab8b01ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:01.227298 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG passlib.handlers.bcrypt [None req-cc899a6f-435f-48e7-8938-9fbaab8b01ca None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81390) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 02 08:55:01.227799 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG passlib.handlers.bcrypt [None req-cc899a6f-435f-48e7-8938-9fbaab8b01ca None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81390) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 02 08:55:01.233441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 6/12] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:01.277107 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-cc899a6f-435f-48e7-8938-9fbaab8b01ca None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:01.278672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-cc899a6f-435f-48e7-8938-9fbaab8b01ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:01.390419 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 7/13] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:55:01.505061 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecf46a-1bc5-4b53-9ab1-26d929de2a61 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:01.529187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:01.607641 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecf46a-1bc5-4b53-9ab1-26d929de2a61 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:01.639844 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c171722-a129-4614-92f0-952bb0537c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:01.640517 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c171722-a129-4614-92f0-952bb0537c97 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:01.640745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c171722-a129-4614-92f0-952bb0537c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:01.641018 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c171722-a129-4614-92f0-952bb0537c97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:01.652157 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c171722-a129-4614-92f0-952bb0537c97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:01.654620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc899a6f-435f-48e7-8938-9fbaab8b01ca None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:01.660873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-3c171722-a129-4614-92f0-952bb0537c97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:55:01.660873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:01.660873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:01.660873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:01.660873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:01.660873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:01.660873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:01.660873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:01.660873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:01.660873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:01.660873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:01.660873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:01.660873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:01.660873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:01.660873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:01.660873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:01.661825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:01.661825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:01.661825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:01.661825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:01.661825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:01.661825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:01.661825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:01.661825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:01.661825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:01.661825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:01.661825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:01.661825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:01.661825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:01.661825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:01.661825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:01.661825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:01.662743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:01.662743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:01.662743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:01.662743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:01.662743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:01.662743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:01.662743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:01.662743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:01.662743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:01.662743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:01.662743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:55:01.662743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:01.662743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 7/14] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:55:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:01.667511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c171722-a129-4614-92f0-952bb0537c97 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:01.750765 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:01.768100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc899a6f-435f-48e7-8938-9fbaab8b01ca None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:01.770436 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:01.771005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c171722-a129-4614-92f0-952bb0537c97 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:01.805053 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03bccb3-6435-4ebf-8896-d8db262f2fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:01.805249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee7732-4d36-4c2d-a6ac-48e9daa4a90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:01.805706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee7732-4d36-4c2d-a6ac-48e9daa4a90b None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:01.805915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bccb3-6435-4ebf-8896-d8db262f2fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:01.806014 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee7732-4d36-4c2d-a6ac-48e9daa4a90b None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:01.806199 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bccb3-6435-4ebf-8896-d8db262f2fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:01.806290 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee7732-4d36-4c2d-a6ac-48e9daa4a90b None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:01.806470 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03bccb3-6435-4ebf-8896-d8db262f2fb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:01.807396 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee7732-4d36-4c2d-a6ac-48e9daa4a90b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:01.807698 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee7732-4d36-4c2d-a6ac-48e9daa4a90b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:01.809085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee7732-4d36-4c2d-a6ac-48e9daa4a90b None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:01.821052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 8/15] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:01.827033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee7732-4d36-4c2d-a6ac-48e9daa4a90b None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:01.876276 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03bccb3-6435-4ebf-8896-d8db262f2fb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:01.877810 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03bccb3-6435-4ebf-8896-d8db262f2fb4 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:01.923369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee7732-4d36-4c2d-a6ac-48e9daa4a90b None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:01.957354 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f633bf-9dff-416e-97f7-54643cf9af09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:01.957973 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f633bf-9dff-416e-97f7-54643cf9af09 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:01.958253 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f633bf-9dff-416e-97f7-54643cf9af09 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:01.958526 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f633bf-9dff-416e-97f7-54643cf9af09 None admin] PATH_INFO: `/v3/users/95efeb6f9f5f497f9453dc5e9764070a` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:01.959670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f633bf-9dff-416e-97f7-54643cf9af09 None admin] RBAC: Authorizing `identity:get_user(user_id=95efeb6f9f5f497f9453dc5e9764070a)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:01.961107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f633bf-9dff-416e-97f7-54643cf9af09 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:01.962833 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 9/16] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:01 2022] GET /identity/v3/users/95efeb6f9f5f497f9453dc5e9764070a => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:01.968014 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f633bf-9dff-416e-97f7-54643cf9af09 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:01.976016 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 8/17] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 02 08:55:01.981334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03bccb3-6435-4ebf-8896-d8db262f2fb4 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:01.982071 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:01.984186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:02.069480 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f633bf-9dff-416e-97f7-54643cf9af09 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:02.089929 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03bccb3-6435-4ebf-8896-d8db262f2fb4 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:02.103871 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef53938-5a0a-422f-bee4-b41e39186fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:02.104497 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef53938-5a0a-422f-bee4-b41e39186fca None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:02.104774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef53938-5a0a-422f-bee4-b41e39186fca None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:02.105040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef53938-5a0a-422f-bee4-b41e39186fca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:02.106135 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef53938-5a0a-422f-bee4-b41e39186fca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:02.107616 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef53938-5a0a-422f-bee4-b41e39186fca None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:02.109128 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 10/18] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:55:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:02.114795 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef53938-5a0a-422f-bee4-b41e39186fca None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:02.123114 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a136c4-95d8-4bd4-a11f-b239f58dfc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:02.123973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a136c4-95d8-4bd4-a11f-b239f58dfc25 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:02.124250 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a136c4-95d8-4bd4-a11f-b239f58dfc25 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:02.124519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a136c4-95d8-4bd4-a11f-b239f58dfc25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:02.143089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a136c4-95d8-4bd4-a11f-b239f58dfc25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:02.144558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a136c4-95d8-4bd4-a11f-b239f58dfc25 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:02.162614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-19a136c4-95d8-4bd4-a11f-b239f58dfc25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:55:02.162614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:02.162614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:02.162614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:02.162614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:02.162614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:02.162614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:02.162614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:02.162614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:02.162614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:02.162614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:02.162614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:02.162614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:55:02.162614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:55:02.162614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:55:02.162614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:55:02.163698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:02.163698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:02.163698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:02.163698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:02.163698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:02.163698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:02.163698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:02.163698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:02.163698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:02.163698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:02.163698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:02.163698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:02.163698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:02.163698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:02.163698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:02.163698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:02.163698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:55:02.164521 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:55:02.164521 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:55:02.164521 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:02.164521 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 9/19] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:55:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:55:02.169422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a136c4-95d8-4bd4-a11f-b239f58dfc25 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:02.192883 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:02.213373 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef53938-5a0a-422f-bee4-b41e39186fca None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:02.226121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:02.247124 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0600b8a-836b-4b75-bd36-bee35bef335c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:02.247720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0600b8a-836b-4b75-bd36-bee35bef335c None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:02.247995 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0600b8a-836b-4b75-bd36-bee35bef335c None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:02.248258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0600b8a-836b-4b75-bd36-bee35bef335c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:02.248976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0600b8a-836b-4b75-bd36-bee35bef335c None admin] RBAC: Adding query filter params (role.id=a0a76de72fba4036bc1e3e64df717c7f, scope.domain.id=default, user.id=95efeb6f9f5f497f9453dc5e9764070a) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:02.249275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0600b8a-836b-4b75-bd36-bee35bef335c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:02.250708 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0600b8a-836b-4b75-bd36-bee35bef335c None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:02.257270 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 11/20] 10.222.0.36 () {56 vars in 1322 bytes} [Fri Sep 2 08:55:02 2022] GET /identity/v3/role_assignments?user.id=95efeb6f9f5f497f9453dc5e9764070a&scope.domain.id=default&role.id=a0a76de72fba4036bc1e3e64df717c7f => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:02.280262 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a136c4-95d8-4bd4-a11f-b239f58dfc25 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:02.317098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c2a68-fc1b-45af-b6b8-ab4a41383a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:02.317716 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c2a68-fc1b-45af-b6b8-ab4a41383a1d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:02.317991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c2a68-fc1b-45af-b6b8-ab4a41383a1d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:02.318260 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c2a68-fc1b-45af-b6b8-ab4a41383a1d None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:02.318916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c2a68-fc1b-45af-b6b8-ab4a41383a1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:02.319253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c2a68-fc1b-45af-b6b8-ab4a41383a1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:02.320665 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c2a68-fc1b-45af-b6b8-ab4a41383a1d None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:02.339311 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 10/21] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:04.352049 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77fdf0-e7cb-485e-93d6-519daee06311 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:04.352587 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77fdf0-e7cb-485e-93d6-519daee06311 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:04.352587 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee77fdf0-e7cb-485e-93d6-519daee06311 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:04.354741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 12/22] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:04.360794 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563655b8-60eb-4427-9f24-f95068c84838 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:04.361639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563655b8-60eb-4427-9f24-f95068c84838 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:04.362103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563655b8-60eb-4427-9f24-f95068c84838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:04.392572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-563655b8-60eb-4427-9f24-f95068c84838 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:04.394096 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-563655b8-60eb-4427-9f24-f95068c84838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:04.484726 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 11/23] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:04.829786 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c7e19-e5ab-40ed-a761-adf47f3a6e45 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:04.830155 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c7e19-e5ab-40ed-a761-adf47f3a6e45 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:04.830344 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c7e19-e5ab-40ed-a761-adf47f3a6e45 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:04.831948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 13/24] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:04.836831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2c823-e3d6-42f8-b282-359330b01c3d None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:04.837119 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2c823-e3d6-42f8-b282-359330b01c3d None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:04.837386 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2c823-e3d6-42f8-b282-359330b01c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:04.864016 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-7ae2c823-e3d6-42f8-b282-359330b01c3d None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:04.865182 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7ae2c823-e3d6-42f8-b282-359330b01c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:04.917803 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:04.945610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 12/25] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:05.055110 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930228e-7713-4b19-89a0-35054009ddb8 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:05.055419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930228e-7713-4b19-89a0-35054009ddb8 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:05.055649 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930228e-7713-4b19-89a0-35054009ddb8 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:05.057656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 14/26] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:05.058523 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2c823-e3d6-42f8-b282-359330b01c3d None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:05.063529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f327c66-d467-47af-84e7-bb02b3e3bb23 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:05.064059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f327c66-d467-47af-84e7-bb02b3e3bb23 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:05.064526 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f327c66-d467-47af-84e7-bb02b3e3bb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:05.084414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-4f327c66-d467-47af-84e7-bb02b3e3bb23 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:05.085985 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-4f327c66-d467-47af-84e7-bb02b3e3bb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:05.159542 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 15/27] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:55:05.161308 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011dc4b2-382e-4312-88f8-3aa2ba52fb09 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:05.165916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011dc4b2-382e-4312-88f8-3aa2ba52fb09 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:05.166519 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011dc4b2-382e-4312-88f8-3aa2ba52fb09 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:05.168545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 16/28] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:05.173999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8f321-0f6e-4c77-b497-4de725dd2e29 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:05.174528 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8f321-0f6e-4c77-b497-4de725dd2e29 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:05.174999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8f321-0f6e-4c77-b497-4de725dd2e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:05.203140 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-80b8f321-0f6e-4c77-b497-4de725dd2e29 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:05.204764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-80b8f321-0f6e-4c77-b497-4de725dd2e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:05.289214 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2c823-e3d6-42f8-b282-359330b01c3d None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:05.345292 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 17/29] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:55:05.346164 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8f321-0f6e-4c77-b497-4de725dd2e29 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:05.353149 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685f1c5-e4e7-4219-aa1a-56ee97260430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:05.354627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685f1c5-e4e7-4219-aa1a-56ee97260430 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:05.355060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685f1c5-e4e7-4219-aa1a-56ee97260430 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:05.356034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685f1c5-e4e7-4219-aa1a-56ee97260430 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:05.356141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:05.368894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6685f1c5-e4e7-4219-aa1a-56ee97260430 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:05.376919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-6685f1c5-e4e7-4219-aa1a-56ee97260430 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:55:05.376919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:05.376919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:05.376919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:05.376919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:05.376919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:05.376919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:05.376919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:05.376919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:05.376919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:05.376919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:05.376919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:05.376919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:05.376919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:05.376919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:05.376919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:05.377995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:05.377995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:05.377995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:05.377995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:05.377995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:05.377995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:05.377995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:05.377995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:05.377995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:05.377995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:05.377995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:05.377995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:05.377995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:05.377995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:05.377995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:05.377995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:05.378996 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:05.378996 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:05.378996 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:05.378996 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:05.378996 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:05.378996 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:05.378996 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:05.378996 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:05.378996 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:05.378996 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:05.378996 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:55:05.378996 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:05.381319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 13/30] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:55:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:05.387208 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685f1c5-e4e7-4219-aa1a-56ee97260430 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:05.459020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:05.521165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8f321-0f6e-4c77-b497-4de725dd2e29 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:05.564220 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64954e0-59d1-4e57-bdd5-e745fdad514e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:05.564980 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64954e0-59d1-4e57-bdd5-e745fdad514e None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:05.565269 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64954e0-59d1-4e57-bdd5-e745fdad514e None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:05.565544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64954e0-59d1-4e57-bdd5-e745fdad514e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:05.578473 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64954e0-59d1-4e57-bdd5-e745fdad514e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:05.580631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64954e0-59d1-4e57-bdd5-e745fdad514e None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:05.582303 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685f1c5-e4e7-4219-aa1a-56ee97260430 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:05.600609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-e64954e0-59d1-4e57-bdd5-e745fdad514e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:55:05.600609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:05.600609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:05.600609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:05.600609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:05.600609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:05.600609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:05.600609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:05.600609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:05.600609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:05.600609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:05.600609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:05.600609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:55:05.600609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:55:05.600609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:55:05.600609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:55:05.601578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:05.601578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:05.601578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:05.601578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:05.601578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:05.601578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:05.601578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:05.601578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:05.601578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:05.601578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:05.601578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:05.601578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:05.601578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:05.601578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:05.601578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:05.601578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:05.601578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:55:05.602413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:55:05.602413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:55:05.602413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:05.602413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 18/31] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:55:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:05.602413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64954e0-59d1-4e57-bdd5-e745fdad514e None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:05.626353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170709ce-05b0-4366-8c9b-1fb8129dac33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:05.627457 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170709ce-05b0-4366-8c9b-1fb8129dac33 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:05.627954 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170709ce-05b0-4366-8c9b-1fb8129dac33 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:05.628416 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170709ce-05b0-4366-8c9b-1fb8129dac33 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:05.629330 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170709ce-05b0-4366-8c9b-1fb8129dac33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:05.629978 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170709ce-05b0-4366-8c9b-1fb8129dac33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:05.631853 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170709ce-05b0-4366-8c9b-1fb8129dac33 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:05.645840 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 14/32] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:05.647046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170709ce-05b0-4366-8c9b-1fb8129dac33 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:05.676917 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:05.732672 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64954e0-59d1-4e57-bdd5-e745fdad514e None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:05.756951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:05.763045 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170709ce-05b0-4366-8c9b-1fb8129dac33 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:05.774481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbf205-66e0-470f-b17c-848dfe2738a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:05.775957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbf205-66e0-470f-b17c-848dfe2738a8 None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:05.776248 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbf205-66e0-470f-b17c-848dfe2738a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:05.776526 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbf205-66e0-470f-b17c-848dfe2738a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:05.777371 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbf205-66e0-470f-b17c-848dfe2738a8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:05.779482 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbf205-66e0-470f-b17c-848dfe2738a8 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:05.799646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3a154-2ec6-4b26-b0c8-c2099057bf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:05.800257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3a154-2ec6-4b26-b0c8-c2099057bf49 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:05.800537 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3a154-2ec6-4b26-b0c8-c2099057bf49 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:05.800806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3a154-2ec6-4b26-b0c8-c2099057bf49 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:05.801449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3a154-2ec6-4b26-b0c8-c2099057bf49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:05.801754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3a154-2ec6-4b26-b0c8-c2099057bf49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:05.803163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e3a154-2ec6-4b26-b0c8-c2099057bf49 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:05.820527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 15/33] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:05.820527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59680103-f149-410c-8dcb-4d71a14fa0f9 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:05.820527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59680103-f149-410c-8dcb-4d71a14fa0f9 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:05.820527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59680103-f149-410c-8dcb-4d71a14fa0f9 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:05.820527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 16/34] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:05.820527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59680103-f149-410c-8dcb-4d71a14fa0f9 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:05.909848 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 19/35] 10.222.0.36 () {60 vars in 1131 bytes} [Fri Sep 2 08:55:05 2022] POST /identity/v3/roles => generated 225 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:55:05.918099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbf205-66e0-470f-b17c-848dfe2738a8 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:05.925478 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59680103-f149-410c-8dcb-4d71a14fa0f9 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:05.975487 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2bd6b2-ede2-4148-9aee-2807a221a5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:05.976139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bd6b2-ede2-4148-9aee-2807a221a5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:05.976414 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bd6b2-ede2-4148-9aee-2807a221a5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:05.976679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bd6b2-ede2-4148-9aee-2807a221a5c4 None admin] PATH_INFO: `/v3/users/95efeb6f9f5f497f9453dc5e9764070a` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:05.978294 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2bd6b2-ede2-4148-9aee-2807a221a5c4 None admin] RBAC: Authorizing `identity:get_user(user_id=95efeb6f9f5f497f9453dc5e9764070a)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:05.979791 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2bd6b2-ede2-4148-9aee-2807a221a5c4 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:05.986245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 17/36] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:05 2022] GET /identity/v3/users/95efeb6f9f5f497f9453dc5e9764070a => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:05.987519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2bd6b2-ede2-4148-9aee-2807a221a5c4 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:06.032215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:06.071724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbf205-66e0-470f-b17c-848dfe2738a8 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:06.091915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:06.124228 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e6e68f-0867-4068-947f-2f7ab49c9d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:06.125605 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6e68f-0867-4068-947f-2f7ab49c9d38 None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:06.125908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6e68f-0867-4068-947f-2f7ab49c9d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:06.126344 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6e68f-0867-4068-947f-2f7ab49c9d38 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:06.127036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e6e68f-0867-4068-947f-2f7ab49c9d38 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:06.128635 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e6e68f-0867-4068-947f-2f7ab49c9d38 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:06.131493 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2bd6b2-ede2-4148-9aee-2807a221a5c4 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:06.147975 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 20/37] 10.222.0.36 () {60 vars in 1131 bytes} [Fri Sep 2 08:55:05 2022] POST /identity/v3/roles => generated 219 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:55:06.149284 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92cc68-247c-4cca-a71b-442c749cba86 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:06.149574 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92cc68-247c-4cca-a71b-442c749cba86 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:06.149856 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d92cc68-247c-4cca-a71b-442c749cba86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:06.171375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-7d92cc68-247c-4cca-a71b-442c749cba86 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:06.172989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-7d92cc68-247c-4cca-a71b-442c749cba86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:06.173582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3740d1-6a89-42c3-8cbd-418a8bb98182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:06.174535 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3740d1-6a89-42c3-8cbd-418a8bb98182 None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:06.175251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3740d1-6a89-42c3-8cbd-418a8bb98182 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:06.175749 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3740d1-6a89-42c3-8cbd-418a8bb98182 None admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:06.176725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3740d1-6a89-42c3-8cbd-418a8bb98182 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:06.178613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3740d1-6a89-42c3-8cbd-418a8bb98182 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:06.252683 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 21/38] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:06.253314 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92cc68-247c-4cca-a71b-442c749cba86 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:06.343406 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 18/39] 10.222.0.36 () {60 vars in 1141 bytes} [Fri Sep 2 08:55:05 2022] POST /identity/v3/projects => generated 301 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:55:06.376640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d92cc68-247c-4cca-a71b-442c749cba86 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:06.385841 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:06.419787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0bd1ea-03b7-4002-979f-39d5a1cd0045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:06.420549 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bd1ea-03b7-4002-979f-39d5a1cd0045 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:06.420838 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bd1ea-03b7-4002-979f-39d5a1cd0045 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:06.421171 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0bd1ea-03b7-4002-979f-39d5a1cd0045 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:06.422444 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0bd1ea-03b7-4002-979f-39d5a1cd0045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:06.423951 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0bd1ea-03b7-4002-979f-39d5a1cd0045 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:06.425806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 22/40] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:55:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:06.432971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3740d1-6a89-42c3-8cbd-418a8bb98182 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:06.523479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:06.552887 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3740d1-6a89-42c3-8cbd-418a8bb98182 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:06.576267 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0bd1ea-03b7-4002-979f-39d5a1cd0045 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:06.596239 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc652b03-bc25-47be-96ab-f6740af5d25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:06.596934 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc652b03-bc25-47be-96ab-f6740af5d25e None admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:06.597223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc652b03-bc25-47be-96ab-f6740af5d25e None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:06.597502 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc652b03-bc25-47be-96ab-f6740af5d25e None admin] PATH_INFO: `/v3/domains/default/users/95efeb6f9f5f497f9453dc5e9764070a/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:06.599767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc652b03-bc25-47be-96ab-f6740af5d25e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=95efeb6f9f5f497f9453dc5e9764070a, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:06.601335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc652b03-bc25-47be-96ab-f6740af5d25e None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:06.616210 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 19/41] 10.222.0.36 () {58 vars in 1361 bytes} [Fri Sep 2 08:55:06 2022] PUT /identity/v3/domains/default/users/95efeb6f9f5f497f9453dc5e9764070a/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:55:06.617542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd5700-f5d4-4060-b308-63964d0f9ace None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:06.617842 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd5700-f5d4-4060-b308-63964d0f9ace None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:06.618109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd5700-f5d4-4060-b308-63964d0f9ace None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:06.619947 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 20/42] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:06.629205 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34164e9-8fb4-43eb-8675-888c23ffa6cc None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:06.629498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34164e9-8fb4-43eb-8675-888c23ffa6cc None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:06.629779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34164e9-8fb4-43eb-8675-888c23ffa6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:06.649902 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-f34164e9-8fb4-43eb-8675-888c23ffa6cc None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:06.651644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-f34164e9-8fb4-43eb-8675-888c23ffa6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:06.695739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0bd1ea-03b7-4002-979f-39d5a1cd0045 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:06.730322 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 21/43] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:55:06.736253 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebd7d9-20ed-458d-89c6-5cb7da060e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:06.736884 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebd7d9-20ed-458d-89c6-5cb7da060e4d None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:06.737169 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebd7d9-20ed-458d-89c6-5cb7da060e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:06.737442 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebd7d9-20ed-458d-89c6-5cb7da060e4d None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:06.738139 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebd7d9-20ed-458d-89c6-5cb7da060e4d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:06.739599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebd7d9-20ed-458d-89c6-5cb7da060e4d None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:06.756533 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 23/44] 10.222.0.36 () {60 vars in 1131 bytes} [Fri Sep 2 08:55:06 2022] POST /identity/v3/roles => generated 223 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:55:07.010948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34164e9-8fb4-43eb-8675-888c23ffa6cc None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:07.082118 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:07.117691 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34164e9-8fb4-43eb-8675-888c23ffa6cc None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:07.154059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eff389-1d53-49ea-8fec-4a71dbb83c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:07.154633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eff389-1d53-49ea-8fec-4a71dbb83c63 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:07.154916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eff389-1d53-49ea-8fec-4a71dbb83c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:07.155234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eff389-1d53-49ea-8fec-4a71dbb83c63 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:07.156335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eff389-1d53-49ea-8fec-4a71dbb83c63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:07.157789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eff389-1d53-49ea-8fec-4a71dbb83c63 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:07.159431 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 22/45] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:55:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:07.165037 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebd7d9-20ed-458d-89c6-5cb7da060e4d None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:07.307619 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:07.374924 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebd7d9-20ed-458d-89c6-5cb7da060e4d None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:07.408328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8263d-b58a-432f-bf71-de8b10f684eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:07.408939 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8263d-b58a-432f-bf71-de8b10f684eb None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:07.409223 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8263d-b58a-432f-bf71-de8b10f684eb None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:07.409517 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8263d-b58a-432f-bf71-de8b10f684eb None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:07.410167 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8263d-b58a-432f-bf71-de8b10f684eb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:07.411661 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8263d-b58a-432f-bf71-de8b10f684eb None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:07.451197 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 24/46] 10.222.0.36 () {60 vars in 1141 bytes} [Fri Sep 2 08:55:07 2022] POST /identity/v3/projects => generated 312 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 08:55:08.564894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf9db5-1665-4102-bc19-f7664c95e002 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:08.565430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf9db5-1665-4102-bc19-f7664c95e002 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:08.565531 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf9db5-1665-4102-bc19-f7664c95e002 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:08.566878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 23/47] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:08.572741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45784371-6618-493e-923b-3c61a29a209a None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:08.573111 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45784371-6618-493e-923b-3c61a29a209a None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:08.573291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45784371-6618-493e-923b-3c61a29a209a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:08.590999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-45784371-6618-493e-923b-3c61a29a209a None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:08.592194 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-45784371-6618-493e-923b-3c61a29a209a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:08.656546 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 25/48] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:08.923403 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf9db5-1665-4102-bc19-f7664c95e002 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:09.022818 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf9db5-1665-4102-bc19-f7664c95e002 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:09.056942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745898b8-5cd1-49fe-b77f-21a19b259a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:09.057572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745898b8-5cd1-49fe-b77f-21a19b259a04 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:09.057858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745898b8-5cd1-49fe-b77f-21a19b259a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:09.058131 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745898b8-5cd1-49fe-b77f-21a19b259a04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:09.060787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:09.069952 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745898b8-5cd1-49fe-b77f-21a19b259a04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:09.074837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-745898b8-5cd1-49fe-b77f-21a19b259a04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:55:09.074837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:09.074837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:09.074837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:09.074837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:09.074837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:09.074837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:09.074837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:09.074837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:09.074837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:09.074837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:09.074837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:09.074837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:09.074837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:09.074837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:09.074837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:09.075725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:09.075725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:09.075725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:09.075725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:09.075725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:09.075725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:09.075725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:09.075725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:09.075725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:09.075725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:09.075725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:09.075725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:09.075725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:09.075725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:09.075725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:09.075725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:09.076798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:09.076798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:09.076798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:09.076798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:09.076798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:09.076798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:09.076798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:09.076798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:09.076798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:09.076798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:09.076798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:55:09.076798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:09.076798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 24/49] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:55:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:09.081657 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45784371-6618-493e-923b-3c61a29a209a None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:09.098754 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:09.179113 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45784371-6618-493e-923b-3c61a29a209a None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:09.212293 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46953d85-7d41-4023-89d8-d8f87d856000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:09.212915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46953d85-7d41-4023-89d8-d8f87d856000 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:09.213193 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46953d85-7d41-4023-89d8-d8f87d856000 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:09.213465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46953d85-7d41-4023-89d8-d8f87d856000 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:09.214213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46953d85-7d41-4023-89d8-d8f87d856000 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:09.214513 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46953d85-7d41-4023-89d8-d8f87d856000 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:09.215922 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46953d85-7d41-4023-89d8-d8f87d856000 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:09.223816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 26/50] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:09.229948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745898b8-5cd1-49fe-b77f-21a19b259a04 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:09.343811 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbae7f7-ed92-428a-862d-38620c9aebe2 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:09.344098 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbae7f7-ed92-428a-862d-38620c9aebe2 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:09.344400 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbae7f7-ed92-428a-862d-38620c9aebe2 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:09.345720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 27/51] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:09.350791 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b67c5c-1c7b-47ea-8d41-680e61d6e4d1 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:09.351163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b67c5c-1c7b-47ea-8d41-680e61d6e4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:09.351509 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b67c5c-1c7b-47ea-8d41-680e61d6e4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:09.356003 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745898b8-5cd1-49fe-b77f-21a19b259a04 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:09.371009 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-30b67c5c-1c7b-47ea-8d41-680e61d6e4d1 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:09.372204 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-30b67c5c-1c7b-47ea-8d41-680e61d6e4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:09.389269 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecde52-ac90-4af4-b417-99450ef28f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:09.389903 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecde52-ac90-4af4-b417-99450ef28f11 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:09.390196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecde52-ac90-4af4-b417-99450ef28f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:09.390475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ecde52-ac90-4af4-b417-99450ef28f11 None admin] PATH_INFO: `/v3/users/95efeb6f9f5f497f9453dc5e9764070a` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:09.391633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ecde52-ac90-4af4-b417-99450ef28f11 None admin] RBAC: Authorizing `identity:get_user(user_id=95efeb6f9f5f497f9453dc5e9764070a)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:09.393090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ecde52-ac90-4af4-b417-99450ef28f11 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:09.394707 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 25/52] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:09 2022] GET /identity/v3/users/95efeb6f9f5f497f9453dc5e9764070a => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:09.400809 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecde52-ac90-4af4-b417-99450ef28f11 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:09.438709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 28/53] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:55:09.463505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:09.501032 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ecde52-ac90-4af4-b417-99450ef28f11 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:09.535148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8ee7d-caa4-40ef-9d2d-e01d44bc9e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:09.535775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8ee7d-caa4-40ef-9d2d-e01d44bc9e44 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:09.536057 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8ee7d-caa4-40ef-9d2d-e01d44bc9e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:09.536331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8ee7d-caa4-40ef-9d2d-e01d44bc9e44 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:09.537448 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8ee7d-caa4-40ef-9d2d-e01d44bc9e44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:09.538906 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8ee7d-caa4-40ef-9d2d-e01d44bc9e44 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:09.541284 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 26/54] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:55:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:09.547028 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b67c5c-1c7b-47ea-8d41-680e61d6e4d1 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:09.557523 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:09.644501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b67c5c-1c7b-47ea-8d41-680e61d6e4d1 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:09.677234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae64e7-7f12-45e5-b91b-9e122d6b60b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:09.677831 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae64e7-7f12-45e5-b91b-9e122d6b60b9 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:09.678110 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae64e7-7f12-45e5-b91b-9e122d6b60b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:09.678382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae64e7-7f12-45e5-b91b-9e122d6b60b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:09.679125 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae64e7-7f12-45e5-b91b-9e122d6b60b9 None admin] RBAC: Adding query filter params (role.id=a0a76de72fba4036bc1e3e64df717c7f, scope.domain.id=default, user.id=95efeb6f9f5f497f9453dc5e9764070a) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:09.679431 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae64e7-7f12-45e5-b91b-9e122d6b60b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:09.680895 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae64e7-7f12-45e5-b91b-9e122d6b60b9 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:09.687083 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 29/55] 10.222.0.36 () {56 vars in 1322 bytes} [Fri Sep 2 08:55:09 2022] GET /identity/v3/role_assignments?user.id=95efeb6f9f5f497f9453dc5e9764070a&scope.domain.id=default&role.id=a0a76de72fba4036bc1e3e64df717c7f => generated 530 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:09.716881 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8ee7d-caa4-40ef-9d2d-e01d44bc9e44 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:09.926734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8ee7d-caa4-40ef-9d2d-e01d44bc9e44 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:09.954987 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:09.960192 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c3337e-096b-4447-b3eb-ebe3c05ea70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:09.960854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3337e-096b-4447-b3eb-ebe3c05ea70b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:09.961139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3337e-096b-4447-b3eb-ebe3c05ea70b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:09.961416 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c3337e-096b-4447-b3eb-ebe3c05ea70b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:09.963183 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3337e-096b-4447-b3eb-ebe3c05ea70b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:09.964620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c3337e-096b-4447-b3eb-ebe3c05ea70b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:09.966184 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 27/56] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:55:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:09.972027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae64e7-7f12-45e5-b91b-9e122d6b60b9 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:10.044297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:10.074133 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae64e7-7f12-45e5-b91b-9e122d6b60b9 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:10.108011 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0371b719-f32f-40bc-bc81-8ac3365e76e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:10.108615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371b719-f32f-40bc-bc81-8ac3365e76e6 None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:10.108899 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371b719-f32f-40bc-bc81-8ac3365e76e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:10.109177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0371b719-f32f-40bc-bc81-8ac3365e76e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:10.109814 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371b719-f32f-40bc-bc81-8ac3365e76e6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:10.111298 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0371b719-f32f-40bc-bc81-8ac3365e76e6 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:10.136955 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 30/57] 10.222.0.36 () {60 vars in 1140 bytes} [Fri Sep 2 08:55:09 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:55:12.008375 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fa0a7-f98c-4943-a9a8-a05dbf874109 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:12.009373 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fa0a7-f98c-4943-a9a8-a05dbf874109 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:12.009464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fa0a7-f98c-4943-a9a8-a05dbf874109 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:12.010768 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 28/58] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:12.015733 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcac73-ef30-4129-9a65-1fd79274dd81 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:12.016166 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcac73-ef30-4129-9a65-1fd79274dd81 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:12.016284 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcac73-ef30-4129-9a65-1fd79274dd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:12.034355 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-84bcac73-ef30-4129-9a65-1fd79274dd81 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:12.035569 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-84bcac73-ef30-4129-9a65-1fd79274dd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:12.099538 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 31/59] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:12.369763 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251fa0a7-f98c-4943-a9a8-a05dbf874109 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:12.468049 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251fa0a7-f98c-4943-a9a8-a05dbf874109 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:12.500037 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389131d-14ac-4096-87c2-117ac8075a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:12.500656 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389131d-14ac-4096-87c2-117ac8075a97 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:12.500940 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389131d-14ac-4096-87c2-117ac8075a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:12.501218 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389131d-14ac-4096-87c2-117ac8075a97 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:12.502336 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389131d-14ac-4096-87c2-117ac8075a97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:12.503796 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389131d-14ac-4096-87c2-117ac8075a97 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:12.505332 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 29/60] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:55:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:12.511380 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcac73-ef30-4129-9a65-1fd79274dd81 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:12.563685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:12.606937 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcac73-ef30-4129-9a65-1fd79274dd81 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:12.639853 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fed7d2-9820-4279-a4a5-b0fbdd277c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:12.640444 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fed7d2-9820-4279-a4a5-b0fbdd277c30 None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:12.640724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fed7d2-9820-4279-a4a5-b0fbdd277c30 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:12.640995 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fed7d2-9820-4279-a4a5-b0fbdd277c30 None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:12.641629 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fed7d2-9820-4279-a4a5-b0fbdd277c30 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:12.643089 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fed7d2-9820-4279-a4a5-b0fbdd277c30 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:12.668447 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 32/61] 10.222.0.36 () {60 vars in 1132 bytes} [Fri Sep 2 08:55:12 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:55:15.571310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9d72b-0e69-49ca-beab-fe092fa8d246 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:15.571310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9d72b-0e69-49ca-beab-fe092fa8d246 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:15.571310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9d72b-0e69-49ca-beab-fe092fa8d246 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:15.572241 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 30/62] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:15.576357 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc96d7c-dba8-4713-b8bc-4aea07fd812a None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:15.577070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc96d7c-dba8-4713-b8bc-4aea07fd812a None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:15.577484 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc96d7c-dba8-4713-b8bc-4aea07fd812a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:15.600034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-0fc96d7c-dba8-4713-b8bc-4aea07fd812a None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:15.601443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0fc96d7c-dba8-4713-b8bc-4aea07fd812a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:15.680512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 33/63] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:15.792435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3040df0-778d-457e-8cc5-b31c27773263 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:15.792940 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3040df0-778d-457e-8cc5-b31c27773263 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:15.793379 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3040df0-778d-457e-8cc5-b31c27773263 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:15.795156 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 31/64] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:15.800827 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f914794-103c-415b-a631-d02ea8faccbd None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:15.801287 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f914794-103c-415b-a631-d02ea8faccbd None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:15.801701 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f914794-103c-415b-a631-d02ea8faccbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:15.820577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-0f914794-103c-415b-a631-d02ea8faccbd None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:15.822469 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0f914794-103c-415b-a631-d02ea8faccbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:15.857818 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:15.891499 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 34/65] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:55:16.066595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f5c6f-9fd2-4fbe-a7f3-4a13033ac846 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:16.067118 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f5c6f-9fd2-4fbe-a7f3-4a13033ac846 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:16.067554 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f5c6f-9fd2-4fbe-a7f3-4a13033ac846 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:16.069289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 32/66] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:16.074771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7d522-1e81-4ada-841a-54757be9f61f None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:16.075263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7d522-1e81-4ada-841a-54757be9f61f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:16.075675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7d522-1e81-4ada-841a-54757be9f61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:16.099190 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-c7a7d522-1e81-4ada-841a-54757be9f61f None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:16.100669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c7a7d522-1e81-4ada-841a-54757be9f61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:16.139198 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f5c6f-9fd2-4fbe-a7f3-4a13033ac846 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:16.185985 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 35/67] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:16.188057 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc0e6f-7d5f-4152-8c26-676e4f1d9148 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:16.188559 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc0e6f-7d5f-4152-8c26-676e4f1d9148 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:16.188891 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc0e6f-7d5f-4152-8c26-676e4f1d9148 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:16.190470 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 36/68] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:16.198335 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef178524-f795-4d41-9797-fb06ffb17ac3 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:16.198827 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef178524-f795-4d41-9797-fb06ffb17ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:16.199315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef178524-f795-4d41-9797-fb06ffb17ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:16.229669 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:16.230673 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-ef178524-f795-4d41-9797-fb06ffb17ac3 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:16.232842 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-ef178524-f795-4d41-9797-fb06ffb17ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:16.276143 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f5c6f-9fd2-4fbe-a7f3-4a13033ac846 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:16.313321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 37/69] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:16.314375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef178524-f795-4d41-9797-fb06ffb17ac3 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:16.330735 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a483d-fb60-4622-a650-df1c66dab8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:16.331328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a483d-fb60-4622-a650-df1c66dab8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:16.331605 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a483d-fb60-4622-a650-df1c66dab8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:16.331875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a483d-fb60-4622-a650-df1c66dab8d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:16.355340 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a483d-fb60-4622-a650-df1c66dab8d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:16.362358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-504a483d-fb60-4622-a650-df1c66dab8d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:16.362358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:16.362358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:16.362358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:16.362358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:16.362358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:16.362358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:16.362358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:16.362358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:16.362358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:16.362358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:16.362358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:16.362358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:16.362358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:16.362358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:16.362358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:16.363248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:16.363248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:16.363248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:16.363248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:16.363248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:16.363248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:16.363248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:16.363248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:16.363248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:16.363248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:16.363248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:16.363248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:16.363248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:16.363248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:16.363248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:16.363248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:16.364012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:16.364012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:16.364012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:16.364012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:16.364012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:16.364012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:16.364012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:16.364012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:16.364012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:16.364012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:16.364012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:16.364012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:16.364611 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 33/70] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:16.373376 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a483d-fb60-4622-a650-df1c66dab8d2 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:16.377138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:16.479378 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef178524-f795-4d41-9797-fb06ffb17ac3 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:16.509037 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a483d-fb60-4622-a650-df1c66dab8d2 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:16.516588 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0012780-4545-4c0c-8944-ad8a3df17b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:16.517244 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0012780-4545-4c0c-8944-ad8a3df17b0d None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:16.517533 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0012780-4545-4c0c-8944-ad8a3df17b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:16.517812 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0012780-4545-4c0c-8944-ad8a3df17b0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:16.528292 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0012780-4545-4c0c-8944-ad8a3df17b0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:16.533146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-f0012780-4545-4c0c-8944-ad8a3df17b0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:55:16.533146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:16.533146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:16.533146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:16.533146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:16.533146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:16.533146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:16.533146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:16.533146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:16.533146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:16.533146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:16.533146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:16.533146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:16.533146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:16.533146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:16.533146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:16.534013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:16.534013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:16.534013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:16.534013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:16.534013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:16.534013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:16.534013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:16.534013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:16.534013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:16.534013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:16.534013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:16.534013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:16.534013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:16.534013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:16.534013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:16.534013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:16.534783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:16.534783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:16.534783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:16.534783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:16.534783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:16.534783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:16.534783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:16.534783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:16.534783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:16.534783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:16.534783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:55:16.534783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:16.534783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 38/71] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:55:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:16.536802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f6817-4a48-4595-bddf-fbaaab415607 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:16.537089 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f6817-4a48-4595-bddf-fbaaab415607 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:16.537367 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f6817-4a48-4595-bddf-fbaaab415607 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:16.543645 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 39/72] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:16.543645 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f6817-4a48-4595-bddf-fbaaab415607 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:16.543829 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:16.552507 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e78746-e9f7-44a0-9052-1a719eb57b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:16.553115 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e78746-e9f7-44a0-9052-1a719eb57b0f None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:16.553402 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e78746-e9f7-44a0-9052-1a719eb57b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:16.553677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e78746-e9f7-44a0-9052-1a719eb57b0f None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:16.554366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e78746-e9f7-44a0-9052-1a719eb57b0f None admin] RBAC: Adding query filter params (name=member) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:16.554670 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e78746-e9f7-44a0-9052-1a719eb57b0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:16.556614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e78746-e9f7-44a0-9052-1a719eb57b0f None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:16.565658 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 34/73] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:16.566426 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e78746-e9f7-44a0-9052-1a719eb57b0f None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:16.641928 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:16.652761 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f6817-4a48-4595-bddf-fbaaab415607 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:16.667421 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e78746-e9f7-44a0-9052-1a719eb57b0f None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:16.686404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4290e-e62d-41de-bef7-0b734009cdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:16.687031 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4290e-e62d-41de-bef7-0b734009cdc9 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:16.687602 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4290e-e62d-41de-bef7-0b734009cdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:16.687875 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4290e-e62d-41de-bef7-0b734009cdc9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:16.697728 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4290e-e62d-41de-bef7-0b734009cdc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:16.701848 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32e50e-45e5-4d4c-8cec-36edbe8b34b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:16.702455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32e50e-45e5-4d4c-8cec-36edbe8b34b4 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:16.702548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-1eb4290e-e62d-41de-bef7-0b734009cdc9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:16.702548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:16.702548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:16.702548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:16.702548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:16.702548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:16.702548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:16.702548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:16.702548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:16.702548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:16.702548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:16.702548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:16.702548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:16.702548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:16.702548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:16.702548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:16.703570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:16.703570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:16.703570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:16.703570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:16.703570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:16.703570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:16.703570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:16.703570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:16.703570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:16.703570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:16.703570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:16.703570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:16.703570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:16.703570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:16.703570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:16.703570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:16.704511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:16.704511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:16.704511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:16.704511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:16.704511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:16.704511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:16.704511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:16.704511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:16.704511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:16.704511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:16.704511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:16.704511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:16.705250 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32e50e-45e5-4d4c-8cec-36edbe8b34b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:16.705250 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32e50e-45e5-4d4c-8cec-36edbe8b34b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:16.705388 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 40/74] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:16.705388 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020140f-976e-4462-b633-5308089d75cf None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:16.705388 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020140f-976e-4462-b633-5308089d75cf None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:16.705388 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e020140f-976e-4462-b633-5308089d75cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:16.707278 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32e50e-45e5-4d4c-8cec-36edbe8b34b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:16.707625 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32e50e-45e5-4d4c-8cec-36edbe8b34b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:16.709022 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c32e50e-45e5-4d4c-8cec-36edbe8b34b4 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:16.717322 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 35/75] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:16.718111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32e50e-45e5-4d4c-8cec-36edbe8b34b4 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:16.722784 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-e020140f-976e-4462-b633-5308089d75cf None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:16.724006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-e020140f-976e-4462-b633-5308089d75cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:16.770463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:16.790920 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 41/76] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:16.791995 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020140f-976e-4462-b633-5308089d75cf None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:16.841135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c32e50e-45e5-4d4c-8cec-36edbe8b34b4 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:16.885227 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298afc7-b683-432f-aa91-03ab39303021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:16.885934 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298afc7-b683-432f-aa91-03ab39303021 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:16.886215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298afc7-b683-432f-aa91-03ab39303021 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:16.886492 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c298afc7-b683-432f-aa91-03ab39303021 None admin] PATH_INFO: `/v3/users/9b90e57a93b44d24bc368ab2e96b81d2` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:16.897490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:16.902468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298afc7-b683-432f-aa91-03ab39303021 None admin] RBAC: Authorizing `identity:get_user(user_id=9b90e57a93b44d24bc368ab2e96b81d2)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:16.903964 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c298afc7-b683-432f-aa91-03ab39303021 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:16.905623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 36/77] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:16 2022] GET /identity/v3/users/9b90e57a93b44d24bc368ab2e96b81d2 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:16.906382 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298afc7-b683-432f-aa91-03ab39303021 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:16.920495 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e020140f-976e-4462-b633-5308089d75cf None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:16.954197 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9f016-7b0d-494e-86d0-f02f585e80ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:16.954829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9f016-7b0d-494e-86d0-f02f585e80ac None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:16.955133 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9f016-7b0d-494e-86d0-f02f585e80ac None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:16.955413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9f016-7b0d-494e-86d0-f02f585e80ac None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:16.965970 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9f016-7b0d-494e-86d0-f02f585e80ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:16.972145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-90d9f016-7b0d-494e-86d0-f02f585e80ac None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:55:16.972145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:16.972145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:16.972145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:16.972145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:16.972145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:16.972145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:16.972145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:16.972145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:16.972145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:16.972145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:16.972145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:16.972145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:16.972145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:16.972145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:16.972145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:16.973259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:16.973259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:16.973259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:16.973259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:16.973259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:16.973259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:16.973259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:16.973259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:16.973259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:16.973259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:16.973259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:16.973259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:16.973259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:16.973259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:16.973259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:16.973259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:16.974123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:16.974123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:16.974123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:16.974123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:16.974123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:16.974123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:16.974123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:16.974123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:16.974123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:16.974123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:16.974123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:55:16.974123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:16.974123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 42/78] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:55:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:16.974837 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9f016-7b0d-494e-86d0-f02f585e80ac None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:17.025900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c298afc7-b683-432f-aa91-03ab39303021 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:17.037517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:17.064280 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff77189-554a-4eea-9335-0d16eab17eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:17.064895 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff77189-554a-4eea-9335-0d16eab17eef None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:17.065180 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff77189-554a-4eea-9335-0d16eab17eef None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:17.065456 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff77189-554a-4eea-9335-0d16eab17eef None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:17.066124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff77189-554a-4eea-9335-0d16eab17eef None admin] RBAC: Adding query filter params (name=member) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:17.066428 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff77189-554a-4eea-9335-0d16eab17eef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:17.067842 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff77189-554a-4eea-9335-0d16eab17eef None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:17.074843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9f016-7b0d-494e-86d0-f02f585e80ac None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:17.075959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 37/79] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:17.077027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff77189-554a-4eea-9335-0d16eab17eef None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:17.109083 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7a97a-8d7a-47e6-ab7b-af083d9175b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:17.109710 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7a97a-8d7a-47e6-ab7b-af083d9175b7 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:17.109995 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7a97a-8d7a-47e6-ab7b-af083d9175b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:17.110274 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7a97a-8d7a-47e6-ab7b-af083d9175b7 None admin] PATH_INFO: `/v3/users/95efeb6f9f5f497f9453dc5e9764070a` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:17.111426 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7a97a-8d7a-47e6-ab7b-af083d9175b7 None admin] RBAC: Authorizing `identity:get_user(user_id=95efeb6f9f5f497f9453dc5e9764070a)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:17.112886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7a97a-8d7a-47e6-ab7b-af083d9175b7 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:17.114461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 43/80] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:16 2022] GET /identity/v3/users/95efeb6f9f5f497f9453dc5e9764070a => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:17.119059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7a97a-8d7a-47e6-ab7b-af083d9175b7 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:17.127684 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:17.178405 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff77189-554a-4eea-9335-0d16eab17eef None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:17.212653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17964d54-188f-4ba7-a3e1-ebc59c1970ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:17.213263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17964d54-188f-4ba7-a3e1-ebc59c1970ec None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:17.213541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17964d54-188f-4ba7-a3e1-ebc59c1970ec None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:17.213815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17964d54-188f-4ba7-a3e1-ebc59c1970ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:17.217328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7a97a-8d7a-47e6-ab7b-af083d9175b7 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:17.250484 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4240d3-989d-4742-aef2-558d456e5d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:17.251104 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4240d3-989d-4742-aef2-558d456e5d43 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:17.251394 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4240d3-989d-4742-aef2-558d456e5d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:17.251668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4240d3-989d-4742-aef2-558d456e5d43 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:17.252325 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4240d3-989d-4742-aef2-558d456e5d43 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:17.252638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4240d3-989d-4742-aef2-558d456e5d43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:17.252768 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:17.254175 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4240d3-989d-4742-aef2-558d456e5d43 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:17.262618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 44/81] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:55:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:17.263428 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4240d3-989d-4742-aef2-558d456e5d43 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:17.279753 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17964d54-188f-4ba7-a3e1-ebc59c1970ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:17.281195 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17964d54-188f-4ba7-a3e1-ebc59c1970ec None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:17.335413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:17.366866 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4240d3-989d-4742-aef2-558d456e5d43 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:17.376690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 38/82] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:17.377601 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17964d54-188f-4ba7-a3e1-ebc59c1970ec None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:17.404252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdc98a-2857-4ea5-82d3-a4a69edf4ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:17.404873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdc98a-2857-4ea5-82d3-a4a69edf4ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:17.405152 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdc98a-2857-4ea5-82d3-a4a69edf4ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:17.405423 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdc98a-2857-4ea5-82d3-a4a69edf4ca6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:17.406504 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdc98a-2857-4ea5-82d3-a4a69edf4ca6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:17.407974 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdc98a-2857-4ea5-82d3-a4a69edf4ca6 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:17.409508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 45/83] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:55:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:17.410254 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdc98a-2857-4ea5-82d3-a4a69edf4ca6 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:17.443869 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:17.473859 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17964d54-188f-4ba7-a3e1-ebc59c1970ec None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:17.509279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48634930-cecf-46c6-8dac-d751aa477deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:17.509910 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48634930-cecf-46c6-8dac-d751aa477deb None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:17.510192 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48634930-cecf-46c6-8dac-d751aa477deb None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:17.510466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48634930-cecf-46c6-8dac-d751aa477deb None admin] PATH_INFO: `/v3/users/9b90e57a93b44d24bc368ab2e96b81d2` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:17.511622 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48634930-cecf-46c6-8dac-d751aa477deb None admin] RBAC: Authorizing `identity:get_user(user_id=9b90e57a93b44d24bc368ab2e96b81d2)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:17.512791 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdc98a-2857-4ea5-82d3-a4a69edf4ca6 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:17.513069 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48634930-cecf-46c6-8dac-d751aa477deb None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:17.514650 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 39/84] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:17 2022] GET /identity/v3/users/9b90e57a93b44d24bc368ab2e96b81d2 => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:17.519904 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48634930-cecf-46c6-8dac-d751aa477deb None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:17.546700 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd0931-3ac3-4907-814d-d8486bf80f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:17.547336 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd0931-3ac3-4907-814d-d8486bf80f58 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:17.547618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd0931-3ac3-4907-814d-d8486bf80f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:17.547897 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcd0931-3ac3-4907-814d-d8486bf80f58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:17.556642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:17.616469 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48634930-cecf-46c6-8dac-d751aa477deb None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:17.616636 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd0931-3ac3-4907-814d-d8486bf80f58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:17.618554 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcd0931-3ac3-4907-814d-d8486bf80f58 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:17.649861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d1291-8068-446a-95e6-26e859f0d032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:17.650481 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d1291-8068-446a-95e6-26e859f0d032 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:17.650766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d1291-8068-446a-95e6-26e859f0d032 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:17.651037 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774d1291-8068-446a-95e6-26e859f0d032 None admin] PATH_INFO: `/v3/users/9b90e57a93b44d24bc368ab2e96b81d2` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:17.652192 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d1291-8068-446a-95e6-26e859f0d032 None admin] RBAC: Authorizing `identity:get_user(user_id=9b90e57a93b44d24bc368ab2e96b81d2)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:17.653627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774d1291-8068-446a-95e6-26e859f0d032 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:17.655213 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 40/85] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:17 2022] GET /identity/v3/users/9b90e57a93b44d24bc368ab2e96b81d2 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:17.655964 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d1291-8068-446a-95e6-26e859f0d032 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:17.668571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:17.720559 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 46/86] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:17.721305 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd0931-3ac3-4907-814d-d8486bf80f58 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:17.757831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774d1291-8068-446a-95e6-26e859f0d032 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:17.759324 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:17.793431 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704563c-02fa-4ab2-a92a-fbb6175f145f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:17.794075 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704563c-02fa-4ab2-a92a-fbb6175f145f None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:17.794357 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704563c-02fa-4ab2-a92a-fbb6175f145f None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:17.794633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704563c-02fa-4ab2-a92a-fbb6175f145f None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:17.795795 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704563c-02fa-4ab2-a92a-fbb6175f145f None admin] RBAC: Authorizing `identity:get_project(project_id=2d3731a0d0214a2ab2046a5d36d29aa4)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:17.797238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704563c-02fa-4ab2-a92a-fbb6175f145f None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:17.798892 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 41/87] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:17 2022] GET /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:17.799707 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704563c-02fa-4ab2-a92a-fbb6175f145f None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:17.824066 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcd0931-3ac3-4907-814d-d8486bf80f58 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:17.857734 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a422057-c21c-45b1-8e99-b674aecd3d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:17.858401 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a422057-c21c-45b1-8e99-b674aecd3d75 None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:17.858796 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a422057-c21c-45b1-8e99-b674aecd3d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:17.859103 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a422057-c21c-45b1-8e99-b674aecd3d75 None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:17.859756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a422057-c21c-45b1-8e99-b674aecd3d75 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:17.861217 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a422057-c21c-45b1-8e99-b674aecd3d75 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:17.873617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:17.881595 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 47/88] 10.222.0.36 () {60 vars in 1132 bytes} [Fri Sep 2 08:55:17 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:55:17.882406 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a422057-c21c-45b1-8e99-b674aecd3d75 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:17.900401 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704563c-02fa-4ab2-a92a-fbb6175f145f None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:17.938296 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb074c-ad68-44f2-b131-943be20f3481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:17.938920 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb074c-ad68-44f2-b131-943be20f3481 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:17.939235 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb074c-ad68-44f2-b131-943be20f3481 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:17.939498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb074c-ad68-44f2-b131-943be20f3481 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:18.015002 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:18.016203 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb074c-ad68-44f2-b131-943be20f3481 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:18.017739 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb074c-ad68-44f2-b131-943be20f3481 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:18.022000 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a422057-c21c-45b1-8e99-b674aecd3d75 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:18.068924 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951f4a4-0412-4ca7-a91f-73b216e3e781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:18.069551 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951f4a4-0412-4ca7-a91f-73b216e3e781 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:18.069857 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951f4a4-0412-4ca7-a91f-73b216e3e781 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:18.070116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951f4a4-0412-4ca7-a91f-73b216e3e781 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:18.101732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:18.143062 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 42/89] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:18.143992 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb074c-ad68-44f2-b131-943be20f3481 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:18.168379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951f4a4-0412-4ca7-a91f-73b216e3e781 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:18.169853 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951f4a4-0412-4ca7-a91f-73b216e3e781 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:18.266819 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb074c-ad68-44f2-b131-943be20f3481 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:18.278468 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:18.287697 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 48/90] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:18.293389 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951f4a4-0412-4ca7-a91f-73b216e3e781 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:18.302534 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3f286-c7de-440d-81b8-0e45cbf06d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:18.303209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3f286-c7de-440d-81b8-0e45cbf06d41 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:18.303493 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3f286-c7de-440d-81b8-0e45cbf06d41 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:18.303765 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3f286-c7de-440d-81b8-0e45cbf06d41 None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:18.304955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3f286-c7de-440d-81b8-0e45cbf06d41 None admin] RBAC: Authorizing `identity:get_project(project_id=2d3731a0d0214a2ab2046a5d36d29aa4)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:18.307010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3f286-c7de-440d-81b8-0e45cbf06d41 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:18.308850 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 43/91] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:18 2022] GET /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:18.309437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3f286-c7de-440d-81b8-0e45cbf06d41 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:18.371778 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:18.434410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951f4a4-0412-4ca7-a91f-73b216e3e781 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:18.439219 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3f286-c7de-440d-81b8-0e45cbf06d41 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:18.481737 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf68735-c0c2-4bde-8975-4dd0525f032a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:18.482438 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf68735-c0c2-4bde-8975-4dd0525f032a None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:18.482723 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf68735-c0c2-4bde-8975-4dd0525f032a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:18.482996 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf68735-c0c2-4bde-8975-4dd0525f032a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:18.483621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea171f0-7e35-4e23-ab3f-409e45078269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:18.483737 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf68735-c0c2-4bde-8975-4dd0525f032a None admin] RBAC: Adding query filter params (role.id=08c2a54a85ef4bb9892a5769ef8a5d95, scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4, user.id=9b90e57a93b44d24bc368ab2e96b81d2) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:18.484044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf68735-c0c2-4bde-8975-4dd0525f032a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:18.484791 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea171f0-7e35-4e23-ab3f-409e45078269 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:18.485271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea171f0-7e35-4e23-ab3f-409e45078269 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:18.485505 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf68735-c0c2-4bde-8975-4dd0525f032a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:18.485936 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea171f0-7e35-4e23-ab3f-409e45078269 None admin] PATH_INFO: `/v3/projects/93bf8f67aa39492d99306ade723efc09` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:18.487400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea171f0-7e35-4e23-ab3f-409e45078269 None admin] RBAC: Authorizing `identity:get_project(project_id=93bf8f67aa39492d99306ade723efc09)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:18.489059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea171f0-7e35-4e23-ab3f-409e45078269 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:18.490924 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 44/92] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:18 2022] GET /identity/v3/projects/93bf8f67aa39492d99306ade723efc09 => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:18.491432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 49/93] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:55:18 2022] GET /identity/v3/role_assignments?user.id=9b90e57a93b44d24bc368ab2e96b81d2&scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4&role.id=08c2a54a85ef4bb9892a5769ef8a5d95 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:18.492260 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea171f0-7e35-4e23-ab3f-409e45078269 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:18.495924 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf68735-c0c2-4bde-8975-4dd0525f032a None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:18.596450 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:18.658957 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea171f0-7e35-4e23-ab3f-409e45078269 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:18.673275 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:18.695228 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf68735-c0c2-4bde-8975-4dd0525f032a None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:18.705823 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168b0b9-c80d-4caf-a263-a037b2e7d5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:18.706599 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168b0b9-c80d-4caf-a263-a037b2e7d5d9 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:18.706894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168b0b9-c80d-4caf-a263-a037b2e7d5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:18.707316 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168b0b9-c80d-4caf-a263-a037b2e7d5d9 None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:18.711052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168b0b9-c80d-4caf-a263-a037b2e7d5d9 None admin] RBAC: Authorizing `identity:get_project(project_id=2d3731a0d0214a2ab2046a5d36d29aa4)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:18.712589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168b0b9-c80d-4caf-a263-a037b2e7d5d9 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:18.714478 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 45/94] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:18 2022] GET /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:18.715093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168b0b9-c80d-4caf-a263-a037b2e7d5d9 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:18.759763 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54efc1bb-e2e6-4055-b393-7d57f3ff0189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:18.761074 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54efc1bb-e2e6-4055-b393-7d57f3ff0189 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:18.761555 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54efc1bb-e2e6-4055-b393-7d57f3ff0189 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:18.762059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54efc1bb-e2e6-4055-b393-7d57f3ff0189 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:18.762984 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54efc1bb-e2e6-4055-b393-7d57f3ff0189 None admin] RBAC: Adding query filter params (role.id=a0a76de72fba4036bc1e3e64df717c7f, scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4, user.id=95efeb6f9f5f497f9453dc5e9764070a) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:18.763510 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54efc1bb-e2e6-4055-b393-7d57f3ff0189 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:18.765161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54efc1bb-e2e6-4055-b393-7d57f3ff0189 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:18.777681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 50/95] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:55:18 2022] GET /identity/v3/role_assignments?user.id=95efeb6f9f5f497f9453dc5e9764070a&scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4&role.id=a0a76de72fba4036bc1e3e64df717c7f => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:18.778770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54efc1bb-e2e6-4055-b393-7d57f3ff0189 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:18.826533 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168b0b9-c80d-4caf-a263-a037b2e7d5d9 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:18.869716 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80199f71-dd74-460d-8555-3db2989df82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:18.870359 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80199f71-dd74-460d-8555-3db2989df82f None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:18.870646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80199f71-dd74-460d-8555-3db2989df82f None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:18.870924 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80199f71-dd74-460d-8555-3db2989df82f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:18.871721 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80199f71-dd74-460d-8555-3db2989df82f None admin] RBAC: Adding query filter params (role.id=08c2a54a85ef4bb9892a5769ef8a5d95, scope.project.id=93bf8f67aa39492d99306ade723efc09, user.id=9b90e57a93b44d24bc368ab2e96b81d2) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:18.872038 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80199f71-dd74-460d-8555-3db2989df82f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:18.873559 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80199f71-dd74-460d-8555-3db2989df82f None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:18.894680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 46/96] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:55:18 2022] GET /identity/v3/role_assignments?user.id=9b90e57a93b44d24bc368ab2e96b81d2&scope.project.id=93bf8f67aa39492d99306ade723efc09&role.id=08c2a54a85ef4bb9892a5769ef8a5d95 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:19.005234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54efc1bb-e2e6-4055-b393-7d57f3ff0189 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:19.041230 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:19.058706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d0e80-6652-4941-bc71-ae7b6aee9b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:19.059359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d0e80-6652-4941-bc71-ae7b6aee9b9f None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:19.059642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d0e80-6652-4941-bc71-ae7b6aee9b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:19.059915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d0e80-6652-4941-bc71-ae7b6aee9b9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:19.060661 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d0e80-6652-4941-bc71-ae7b6aee9b9f None admin] RBAC: Adding query filter params (role.id=9c6920b8e6f744a6a80371b4218605fd, scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4, user.id=9b90e57a93b44d24bc368ab2e96b81d2) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:19.060968 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d0e80-6652-4941-bc71-ae7b6aee9b9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:19.062684 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d0e80-6652-4941-bc71-ae7b6aee9b9f None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:19.071934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 51/97] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:55:18 2022] GET /identity/v3/role_assignments?user.id=9b90e57a93b44d24bc368ab2e96b81d2&scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4&role.id=9c6920b8e6f744a6a80371b4218605fd => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:20.303947 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c82ab-8ca3-4537-85b8-760fb074ae2c None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:20.304460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c82ab-8ca3-4537-85b8-760fb074ae2c None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:20.304564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c82ab-8ca3-4537-85b8-760fb074ae2c None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:20.306056 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 47/98] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:20.317183 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d289226-f176-4a04-b26b-a47d2fc7cbb6 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:20.317183 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d289226-f176-4a04-b26b-a47d2fc7cbb6 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:20.317183 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d289226-f176-4a04-b26b-a47d2fc7cbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:20.339958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-0d289226-f176-4a04-b26b-a47d2fc7cbb6 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:20.341055 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0d289226-f176-4a04-b26b-a47d2fc7cbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:20.441145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 52/99] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:20.733647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03085e-5b34-43df-bda3-4a402adfc763 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:20.733965 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03085e-5b34-43df-bda3-4a402adfc763 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:20.734314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03085e-5b34-43df-bda3-4a402adfc763 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:20.735685 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 48/100] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:20.740875 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc3143-4063-4794-9d93-f91abca74193 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:20.741167 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc3143-4063-4794-9d93-f91abca74193 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:20.741449 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc3143-4063-4794-9d93-f91abca74193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:20.772074 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-2fbc3143-4063-4794-9d93-f91abca74193 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:20.773275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-2fbc3143-4063-4794-9d93-f91abca74193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:20.860555 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 53/101] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:20.875139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03085e-5b34-43df-bda3-4a402adfc763 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:20.939791 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:21.008327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03085e-5b34-43df-bda3-4a402adfc763 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:21.056943 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4697d741-d4b8-4a9c-8ee9-c0352cd93b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:21.057597 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697d741-d4b8-4a9c-8ee9-c0352cd93b4c None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:21.057889 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697d741-d4b8-4a9c-8ee9-c0352cd93b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:21.058158 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697d741-d4b8-4a9c-8ee9-c0352cd93b4c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:21.080187 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4697d741-d4b8-4a9c-8ee9-c0352cd93b4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:21.085204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-4697d741-d4b8-4a9c-8ee9-c0352cd93b4c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 08:55:21.085204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:21.085204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:21.085204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:21.085204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:21.085204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:21.085204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:21.085204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:21.085204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:21.085204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:21.085204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:21.085204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:21.085204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:21.085204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:21.085204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:21.085204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:21.086098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:21.086098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:21.086098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:21.086098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:21.086098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:21.086098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:21.086098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:21.086098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:21.086098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:21.086098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:21.086098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:21.086098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:21.086098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:21.086098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:21.086098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:21.086098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:21.087483 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:21.087483 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:21.087483 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:21.087483 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:21.087483 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:21.087483 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:21.087483 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:21.087483 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:21.087483 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:21.087483 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:21.087483 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 08:55:21.087483 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:21.087483 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 49/102] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:21.092267 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc3143-4063-4794-9d93-f91abca74193 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:21.178202 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:21.254325 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbc3143-4063-4794-9d93-f91abca74193 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:21.300906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8c822-a78e-4468-8894-ac8d588b2bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:21.301447 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8c822-a78e-4468-8894-ac8d588b2bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:21.301787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8c822-a78e-4468-8894-ac8d588b2bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:21.302113 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8c822-a78e-4468-8894-ac8d588b2bd5 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:21.302789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8c822-a78e-4468-8894-ac8d588b2bd5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:21.303108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8c822-a78e-4468-8894-ac8d588b2bd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:21.304509 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8c822-a78e-4468-8894-ac8d588b2bd5 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:21.318791 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 54/103] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:21.322320 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb21102-b786-4455-af73-b8b39966a258 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:21.322729 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb21102-b786-4455-af73-b8b39966a258 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:21.322890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb21102-b786-4455-af73-b8b39966a258 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:21.324300 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 50/104] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:21.328175 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb21102-b786-4455-af73-b8b39966a258 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:21.332101 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8c822-a78e-4468-8894-ac8d588b2bd5 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:21.471042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb21102-b786-4455-af73-b8b39966a258 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:21.478158 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8c822-a78e-4468-8894-ac8d588b2bd5 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:21.479595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:21.487399 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:21.515719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ddc89-439b-4728-b626-38be4eb81825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:21.517372 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ddc89-439b-4728-b626-38be4eb81825 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:21.517658 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ddc89-439b-4728-b626-38be4eb81825 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:21.517935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ddc89-439b-4728-b626-38be4eb81825 None admin] PATH_INFO: `/v3/users/6ac5aac6d9b8480b9ec17074a26879ad` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:21.532093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e3542-e4dd-4029-b14d-c8bfca69fe3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:21.532737 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e3542-e4dd-4029-b14d-c8bfca69fe3b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:21.533019 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e3542-e4dd-4029-b14d-c8bfca69fe3b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:21.533290 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e3542-e4dd-4029-b14d-c8bfca69fe3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:21.535560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e3542-e4dd-4029-b14d-c8bfca69fe3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:21.537089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e3542-e4dd-4029-b14d-c8bfca69fe3b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:21.538678 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 51/105] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:55:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:21.539920 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce5126-3955-4e19-bff5-449bbc539d56 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:21.540175 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce5126-3955-4e19-bff5-449bbc539d56 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:21.540450 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce5126-3955-4e19-bff5-449bbc539d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:21.548361 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ddc89-439b-4728-b626-38be4eb81825 None admin] RBAC: Authorizing `identity:get_user(user_id=6ac5aac6d9b8480b9ec17074a26879ad)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:21.549836 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ddc89-439b-4728-b626-38be4eb81825 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:21.553591 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 55/106] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:21 2022] GET /identity/v3/users/6ac5aac6d9b8480b9ec17074a26879ad => generated 289 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:21.554293 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ddc89-439b-4728-b626-38be4eb81825 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:21.582096 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-00ce5126-3955-4e19-bff5-449bbc539d56 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:21.583324 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-00ce5126-3955-4e19-bff5-449bbc539d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:21.652850 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 52/107] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:21.653821 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce5126-3955-4e19-bff5-449bbc539d56 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:21.667395 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ddc89-439b-4728-b626-38be4eb81825 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:21.704536 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0547c7-2b59-4cc6-8dce-7447a5439520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:21.705158 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0547c7-2b59-4cc6-8dce-7447a5439520 None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:21.705442 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0547c7-2b59-4cc6-8dce-7447a5439520 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:21.705716 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0547c7-2b59-4cc6-8dce-7447a5439520 None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:21.706372 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0547c7-2b59-4cc6-8dce-7447a5439520 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:21.707976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0547c7-2b59-4cc6-8dce-7447a5439520 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:21.748347 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 56/108] 10.222.0.36 () {60 vars in 1141 bytes} [Fri Sep 2 08:55:21 2022] POST /identity/v3/projects => generated 302 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:55:21.749623 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84cc91-ee84-457f-a4f1-78deb68bd842 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:21.749969 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84cc91-ee84-457f-a4f1-78deb68bd842 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:21.750253 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84cc91-ee84-457f-a4f1-78deb68bd842 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:21.751693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 57/109] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:21.769482 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce00c72-023d-4ff7-a575-dbfafd07e20a None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:21.769783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce00c72-023d-4ff7-a575-dbfafd07e20a None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:21.775099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce00c72-023d-4ff7-a575-dbfafd07e20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:21.812705 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce5126-3955-4e19-bff5-449bbc539d56 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:21.816779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:21.831106 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-8ce00c72-023d-4ff7-a575-dbfafd07e20a None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:21.832591 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8ce00c72-023d-4ff7-a575-dbfafd07e20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:21.872196 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:21.936747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb71f5-927a-4a38-8049-ae9db624afdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:21.937630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb71f5-927a-4a38-8049-ae9db624afdd None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:21.937915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb71f5-927a-4a38-8049-ae9db624afdd None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:21.938192 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb71f5-927a-4a38-8049-ae9db624afdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:21.985945 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 58/110] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 02 08:55:21.986815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce00c72-023d-4ff7-a575-dbfafd07e20a None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:22.066419 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb71f5-927a-4a38-8049-ae9db624afdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:22.068715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eb71f5-927a-4a38-8049-ae9db624afdd None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:22.110179 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce00c72-023d-4ff7-a575-dbfafd07e20a None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:22.157228 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dcef8c-eb8b-408e-9556-a3f9786d3009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:22.157873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcef8c-eb8b-408e-9556-a3f9786d3009 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:22.158256 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcef8c-eb8b-408e-9556-a3f9786d3009 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:22.158533 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcef8c-eb8b-408e-9556-a3f9786d3009 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:22.171630 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dcef8c-eb8b-408e-9556-a3f9786d3009 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:22.176586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-a8dcef8c-eb8b-408e-9556-a3f9786d3009 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:22.176586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:22.176586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:22.176586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:22.176586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:22.176586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:22.176586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:22.176586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:22.176586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:22.176586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:22.176586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:22.176586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:22.176586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:22.176586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:22.176586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:22.176586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:22.178445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:22.178445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:22.178445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:22.178445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:22.178445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:22.178445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:22.178445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:22.178445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:22.178445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:22.178445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:22.178445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:22.178445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:22.178445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:22.178445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:22.178445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:22.178445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:22.179276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:22.179276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:22.179276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:22.179276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:22.179276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:22.179276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:22.179276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:22.179276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:22.179276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:22.179276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:22.179276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:22.179276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:22.179276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 59/111] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:22.179276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b52020-0252-4211-b55f-cdc7f77570c8 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:22.180766 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b52020-0252-4211-b55f-cdc7f77570c8 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:22.180766 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b52020-0252-4211-b55f-cdc7f77570c8 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:22.180766 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 60/112] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:22.181561 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2428b0-ff86-4f5b-ac7b-243aeb85c521 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:22.182344 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2428b0-ff86-4f5b-ac7b-243aeb85c521 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:22.186647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2428b0-ff86-4f5b-ac7b-243aeb85c521 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:22.188592 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 61/113] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:22.193408 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68777e-d464-40b6-84cb-90bb967eb0c1 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:22.193408 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68777e-d464-40b6-84cb-90bb967eb0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:22.193408 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68777e-d464-40b6-84cb-90bb967eb0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:22.209609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-db68777e-d464-40b6-84cb-90bb967eb0c1 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:22.210898 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-db68777e-d464-40b6-84cb-90bb967eb0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:22.213537 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:22.216896 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 53/114] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:22.217961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb71f5-927a-4a38-8049-ae9db624afdd None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:22.220447 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:22.281833 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 62/115] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:22.283465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b49a0-bfc9-4b24-b5b2-fcafb30832e8 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:22.283972 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b49a0-bfc9-4b24-b5b2-fcafb30832e8 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:22.284479 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b49a0-bfc9-4b24-b5b2-fcafb30832e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:22.303231 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-d58b49a0-bfc9-4b24-b5b2-fcafb30832e8 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:22.304710 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-d58b49a0-bfc9-4b24-b5b2-fcafb30832e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:22.344815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb71f5-927a-4a38-8049-ae9db624afdd None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:22.380869 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b23f9-a8af-4e47-a28e-07ef8f98692e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:22.381787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b23f9-a8af-4e47-a28e-07ef8f98692e None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:22.382077 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b23f9-a8af-4e47-a28e-07ef8f98692e None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:22.382490 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b23f9-a8af-4e47-a28e-07ef8f98692e None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:22.383472 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b23f9-a8af-4e47-a28e-07ef8f98692e None admin] RBAC: Adding query filter params (name=member) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:22.383782 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b23f9-a8af-4e47-a28e-07ef8f98692e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:22.385326 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b23f9-a8af-4e47-a28e-07ef8f98692e None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:22.393579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 63/116] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:22.395216 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 54/117] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:22.395577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b49a0-bfc9-4b24-b5b2-fcafb30832e8 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:22.395958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b23f9-a8af-4e47-a28e-07ef8f98692e None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:22.524776 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:22.527641 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:22.575494 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b49a0-bfc9-4b24-b5b2-fcafb30832e8 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:22.595966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b23f9-a8af-4e47-a28e-07ef8f98692e None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:22.617395 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25bc5a-a678-4649-aa67-974f45239b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:22.618188 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25bc5a-a678-4649-aa67-974f45239b9a None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:22.618475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25bc5a-a678-4649-aa67-974f45239b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:22.618748 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25bc5a-a678-4649-aa67-974f45239b9a None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:22.620437 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25bc5a-a678-4649-aa67-974f45239b9a None admin] RBAC: Authorizing `identity:get_project(project_id=2d3731a0d0214a2ab2046a5d36d29aa4)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:22.622341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25bc5a-a678-4649-aa67-974f45239b9a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:22.624435 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 64/118] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:22 2022] GET /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:22.627838 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25bc5a-a678-4649-aa67-974f45239b9a None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:22.634547 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04fc0d-b202-409f-ae77-6cef376a8ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:22.635219 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04fc0d-b202-409f-ae77-6cef376a8ebb None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:22.635550 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04fc0d-b202-409f-ae77-6cef376a8ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:22.635829 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d04fc0d-b202-409f-ae77-6cef376a8ebb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:22.646855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d04fc0d-b202-409f-ae77-6cef376a8ebb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:22.652973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-0d04fc0d-b202-409f-ae77-6cef376a8ebb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:22.652973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:22.652973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:22.652973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:22.652973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:22.652973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:22.652973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:22.652973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:22.652973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:22.652973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:22.652973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:22.652973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:22.652973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:22.652973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:22.652973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:22.652973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:22.653806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:22.653806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:22.653806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:22.653806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:22.653806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:22.653806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:22.653806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:22.653806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:22.653806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:22.653806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:22.653806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:22.653806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:22.653806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:22.653806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:22.653806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:22.653806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:22.654573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:22.654573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:22.654573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:22.654573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:22.654573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:22.654573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:22.654573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:22.654573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:22.654573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:22.654573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:22.654573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:22.654573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:22.654573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 55/119] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:22.655235 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04fc0d-b202-409f-ae77-6cef376a8ebb None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:22.752329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25bc5a-a678-4649-aa67-974f45239b9a None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:22.766172 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d04fc0d-b202-409f-ae77-6cef376a8ebb None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:22.787582 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c11f3-2448-47ab-a1e2-d922e87413d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:22.788226 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c11f3-2448-47ab-a1e2-d922e87413d6 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:22.788496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c11f3-2448-47ab-a1e2-d922e87413d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:22.788764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996c11f3-2448-47ab-a1e2-d922e87413d6 None admin] PATH_INFO: `/v3/users/9b90e57a93b44d24bc368ab2e96b81d2` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:22.790477 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c11f3-2448-47ab-a1e2-d922e87413d6 None admin] RBAC: Authorizing `identity:get_user(user_id=9b90e57a93b44d24bc368ab2e96b81d2)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:22.791962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996c11f3-2448-47ab-a1e2-d922e87413d6 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:22.792364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:22.793554 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 65/120] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:22 2022] GET /identity/v3/users/9b90e57a93b44d24bc368ab2e96b81d2 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:22.798186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c11f3-2448-47ab-a1e2-d922e87413d6 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:22.802801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:22.808773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b617d-8b65-4418-b93b-4a198d6b7f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:22.809327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b617d-8b65-4418-b93b-4a198d6b7f8f None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:22.809607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b617d-8b65-4418-b93b-4a198d6b7f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:22.809879 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b617d-8b65-4418-b93b-4a198d6b7f8f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:22.822836 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b617d-8b65-4418-b93b-4a198d6b7f8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:22.827679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-d99b617d-8b65-4418-b93b-4a198d6b7f8f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:55:22.827679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:22.827679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:22.827679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:22.827679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:22.827679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:22.827679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:22.827679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:22.827679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:22.827679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:22.827679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:22.827679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:22.827679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:22.827679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:22.827679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:22.827679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:22.828513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:22.828513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:22.828513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:22.828513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:22.828513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:22.828513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:22.828513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:22.828513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:22.828513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:22.828513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:22.828513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:22.828513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:22.828513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:22.828513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:22.828513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:22.828513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:22.829463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:22.829463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:22.829463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:22.829463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:22.829463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:22.829463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:22.829463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:22.829463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:22.829463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:22.829463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:22.829463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:55:22.829463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:22.829463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 56/121] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:55:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:22.831839 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b617d-8b65-4418-b93b-4a198d6b7f8f None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:22.906366 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996c11f3-2448-47ab-a1e2-d922e87413d6 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:22.938094 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b617d-8b65-4418-b93b-4a198d6b7f8f None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:22.943508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef5fb4-4f74-49d8-950c-50b82dba1c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:22.944116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef5fb4-4f74-49d8-950c-50b82dba1c05 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:22.944400 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef5fb4-4f74-49d8-950c-50b82dba1c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:22.944677 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef5fb4-4f74-49d8-950c-50b82dba1c05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:22.945433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef5fb4-4f74-49d8-950c-50b82dba1c05 None admin] RBAC: Adding query filter params (role.id=b1bd3fa77e3d4c83aa57374cb35fd21f, scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4, user.id=6ac5aac6d9b8480b9ec17074a26879ad) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:22.945744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef5fb4-4f74-49d8-950c-50b82dba1c05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:22.947219 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef5fb4-4f74-49d8-950c-50b82dba1c05 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:22.952631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 66/122] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:55:22 2022] GET /identity/v3/role_assignments?user.id=6ac5aac6d9b8480b9ec17074a26879ad&scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4&role.id=b1bd3fa77e3d4c83aa57374cb35fd21f => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:22.953403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef5fb4-4f74-49d8-950c-50b82dba1c05 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:22.986883 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce1431-588d-4a23-a5c1-25e2a5828140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:22.987913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce1431-588d-4a23-a5c1-25e2a5828140 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:22.988349 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce1431-588d-4a23-a5c1-25e2a5828140 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:22.988761 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ce1431-588d-4a23-a5c1-25e2a5828140 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:22.989746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce1431-588d-4a23-a5c1-25e2a5828140 None admin] RBAC: Adding query filter params (name=member) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:22.990198 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce1431-588d-4a23-a5c1-25e2a5828140 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:22.991988 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ce1431-588d-4a23-a5c1-25e2a5828140 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:23.003944 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 57/123] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:23.009713 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce1431-588d-4a23-a5c1-25e2a5828140 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:23.034682 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:23.072577 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:23.080294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef5fb4-4f74-49d8-950c-50b82dba1c05 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:23.122893 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7281d-4b9e-4499-9f33-b8e8c86f9a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:23.123935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7281d-4b9e-4499-9f33-b8e8c86f9a84 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:23.124408 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7281d-4b9e-4499-9f33-b8e8c86f9a84 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:23.124860 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7281d-4b9e-4499-9f33-b8e8c86f9a84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:23.137733 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e7281d-4b9e-4499-9f33-b8e8c86f9a84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:23.140992 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ce1431-588d-4a23-a5c1-25e2a5828140 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:23.143861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-a9e7281d-4b9e-4499-9f33-b8e8c86f9a84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:55:23.143861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:23.143861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:23.143861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:23.143861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:23.143861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:23.143861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:23.143861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:23.143861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:23.143861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:23.143861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:23.143861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:23.143861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:23.143861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:23.143861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:23.143861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:23.144719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:23.144719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:23.144719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:23.144719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:23.144719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:23.144719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:23.144719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:23.144719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:23.144719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:23.144719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:23.144719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:23.144719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:23.144719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:23.144719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:23.144719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:23.144719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:23.145492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:23.145492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:23.145492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:23.145492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:23.145492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:23.145492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:23.145492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:23.145492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:23.145492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:23.145492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:23.145492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:55:23.145492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:23.147811 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 67/124] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:55:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:23.149327 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7281d-4b9e-4499-9f33-b8e8c86f9a84 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:23.187453 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f817dfd-362a-409e-9dbd-20f649a550f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:23.188026 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f817dfd-362a-409e-9dbd-20f649a550f5 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:23.188307 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f817dfd-362a-409e-9dbd-20f649a550f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:23.188578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f817dfd-362a-409e-9dbd-20f649a550f5 None admin] PATH_INFO: `/v3/projects/93bf8f67aa39492d99306ade723efc09` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:23.189765 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f817dfd-362a-409e-9dbd-20f649a550f5 None admin] RBAC: Authorizing `identity:get_project(project_id=93bf8f67aa39492d99306ade723efc09)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:23.191483 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f817dfd-362a-409e-9dbd-20f649a550f5 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:23.193286 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 58/125] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:23 2022] GET /identity/v3/projects/93bf8f67aa39492d99306ade723efc09 => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:23.193873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f817dfd-362a-409e-9dbd-20f649a550f5 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:23.279742 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7281d-4b9e-4499-9f33-b8e8c86f9a84 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:23.302931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f817dfd-362a-409e-9dbd-20f649a550f5 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:23.307481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:23.316157 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3d3b0-0a5b-4f0e-ae22-d48648a95f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:23.317060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3d3b0-0a5b-4f0e-ae22-d48648a95f5b None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:23.317561 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3d3b0-0a5b-4f0e-ae22-d48648a95f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:23.318076 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:23.318405 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3d3b0-0a5b-4f0e-ae22-d48648a95f5b None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:23.319518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3d3b0-0a5b-4f0e-ae22-d48648a95f5b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:23.320026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3d3b0-0a5b-4f0e-ae22-d48648a95f5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:23.322250 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3d3b0-0a5b-4f0e-ae22-d48648a95f5b None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:23.331878 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 68/126] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:55:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:23.337531 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3d3b0-0a5b-4f0e-ae22-d48648a95f5b None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:23.347809 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f650b-6681-4ca2-996d-f381e21864f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:23.349005 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f650b-6681-4ca2-996d-f381e21864f5 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:23.349484 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f650b-6681-4ca2-996d-f381e21864f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:23.349917 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f650b-6681-4ca2-996d-f381e21864f5 None admin] PATH_INFO: `/v3/users/9b90e57a93b44d24bc368ab2e96b81d2` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:23.356745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f650b-6681-4ca2-996d-f381e21864f5 None admin] RBAC: Authorizing `identity:get_user(user_id=9b90e57a93b44d24bc368ab2e96b81d2)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:23.358379 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6f650b-6681-4ca2-996d-f381e21864f5 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:23.360256 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 59/127] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:23 2022] GET /identity/v3/users/9b90e57a93b44d24bc368ab2e96b81d2 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:23.361219 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f650b-6681-4ca2-996d-f381e21864f5 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:23.441216 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3d3b0-0a5b-4f0e-ae22-d48648a95f5b None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:23.469207 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f650b-6681-4ca2-996d-f381e21864f5 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:23.477263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5db1fa-c044-44f5-bd29-93f815455813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:23.477732 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5db1fa-c044-44f5-bd29-93f815455813 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:23.478115 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5db1fa-c044-44f5-bd29-93f815455813 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:23.478504 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5db1fa-c044-44f5-bd29-93f815455813 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:23.479326 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5db1fa-c044-44f5-bd29-93f815455813 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:23.479729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5db1fa-c044-44f5-bd29-93f815455813 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:23.481212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5db1fa-c044-44f5-bd29-93f815455813 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:23.494315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 69/128] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:23.494315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5db1fa-c044-44f5-bd29-93f815455813 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:23.514802 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c98cd-d2be-4893-87f0-6514cb15dc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:23.515503 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c98cd-d2be-4893-87f0-6514cb15dc47 None admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:23.515789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c98cd-d2be-4893-87f0-6514cb15dc47 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:23.516066 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c98cd-d2be-4893-87f0-6514cb15dc47 None admin] PATH_INFO: `/v3/projects/93bf8f67aa39492d99306ade723efc09/users/9b90e57a93b44d24bc368ab2e96b81d2/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:23.518145 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c98cd-d2be-4893-87f0-6514cb15dc47 None admin] RBAC: Authorizing `identity:create_grant(project_id=93bf8f67aa39492d99306ade723efc09, user_id=9b90e57a93b44d24bc368ab2e96b81d2, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:23.519651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748c98cd-d2be-4893-87f0-6514cb15dc47 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:23.543093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 60/129] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:55:23 2022] PUT /identity/v3/projects/93bf8f67aa39492d99306ade723efc09/users/9b90e57a93b44d24bc368ab2e96b81d2/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:55:23.543093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c98cd-d2be-4893-87f0-6514cb15dc47 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:23.570909 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:23.630866 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5db1fa-c044-44f5-bd29-93f815455813 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:23.670032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc58f46-1b0f-463f-91f8-0dec86f19857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:23.670744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc58f46-1b0f-463f-91f8-0dec86f19857 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:23.671033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc58f46-1b0f-463f-91f8-0dec86f19857 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:23.671344 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc58f46-1b0f-463f-91f8-0dec86f19857 None admin] PATH_INFO: `/v3/users/9b90e57a93b44d24bc368ab2e96b81d2` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:23.672554 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc58f46-1b0f-463f-91f8-0dec86f19857 None admin] RBAC: Authorizing `identity:get_user(user_id=9b90e57a93b44d24bc368ab2e96b81d2)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:23.674161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc58f46-1b0f-463f-91f8-0dec86f19857 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:23.675996 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 70/130] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:23 2022] GET /identity/v3/users/9b90e57a93b44d24bc368ab2e96b81d2 => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:23.676698 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc58f46-1b0f-463f-91f8-0dec86f19857 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:23.679592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:23.781484 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c98cd-d2be-4893-87f0-6514cb15dc47 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:23.802087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc58f46-1b0f-463f-91f8-0dec86f19857 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:23.825098 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:23.842579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d82a33-3d21-486a-a7ea-22368f4ce8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:23.843612 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d82a33-3d21-486a-a7ea-22368f4ce8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:23.844092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d82a33-3d21-486a-a7ea-22368f4ce8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:23.844671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d82a33-3d21-486a-a7ea-22368f4ce8f1 None admin] PATH_INFO: `/v3/users/95efeb6f9f5f497f9453dc5e9764070a` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:23.846123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d82a33-3d21-486a-a7ea-22368f4ce8f1 None admin] RBAC: Authorizing `identity:get_user(user_id=95efeb6f9f5f497f9453dc5e9764070a)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:23.847834 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d82a33-3d21-486a-a7ea-22368f4ce8f1 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:23.849601 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 71/131] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:23 2022] GET /identity/v3/users/95efeb6f9f5f497f9453dc5e9764070a => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:23.850576 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d82a33-3d21-486a-a7ea-22368f4ce8f1 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:23.887690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a543a-a28b-4660-a320-b656ced1783d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:23.888399 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a543a-a28b-4660-a320-b656ced1783d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:23.888736 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a543a-a28b-4660-a320-b656ced1783d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:23.889022 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a543a-a28b-4660-a320-b656ced1783d None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:23.891291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a543a-a28b-4660-a320-b656ced1783d None admin] RBAC: Authorizing `identity:get_project(project_id=2d3731a0d0214a2ab2046a5d36d29aa4)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:23.892774 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a543a-a28b-4660-a320-b656ced1783d None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:23.894455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 61/132] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:23 2022] GET /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4 => generated 298 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:23.895255 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a543a-a28b-4660-a320-b656ced1783d None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:23.997269 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d82a33-3d21-486a-a7ea-22368f4ce8f1 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:24.032020 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e22e5-8226-4142-bd4f-50674825d35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:24.032587 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e22e5-8226-4142-bd4f-50674825d35c None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:24.032876 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e22e5-8226-4142-bd4f-50674825d35c None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:24.033155 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e22e5-8226-4142-bd4f-50674825d35c None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:24.034291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e22e5-8226-4142-bd4f-50674825d35c None admin] RBAC: Authorizing `identity:get_project(project_id=2d3731a0d0214a2ab2046a5d36d29aa4)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:24.035774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e22e5-8226-4142-bd4f-50674825d35c None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:24.037560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 72/133] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:23 2022] GET /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:24.038126 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e22e5-8226-4142-bd4f-50674825d35c None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:24.064263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a543a-a28b-4660-a320-b656ced1783d None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:24.079163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:24.110684 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:24.116198 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2957b32-19ec-4c12-a292-111e6c2081f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:24.116857 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2957b32-19ec-4c12-a292-111e6c2081f8 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:24.117165 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2957b32-19ec-4c12-a292-111e6c2081f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:24.117455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2957b32-19ec-4c12-a292-111e6c2081f8 None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:24.118664 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2957b32-19ec-4c12-a292-111e6c2081f8 None admin] RBAC: Authorizing `identity:get_project(project_id=2d3731a0d0214a2ab2046a5d36d29aa4)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:24.120259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2957b32-19ec-4c12-a292-111e6c2081f8 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:24.121912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 62/134] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:23 2022] GET /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:24.123214 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19474b-afe4-4d3a-a17c-00d9b8841eec None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:24.123472 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19474b-afe4-4d3a-a17c-00d9b8841eec None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:24.123750 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca19474b-afe4-4d3a-a17c-00d9b8841eec None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:24.129523 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 63/135] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:24.130305 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca19474b-afe4-4d3a-a17c-00d9b8841eec None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:24.160799 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e22e5-8226-4142-bd4f-50674825d35c None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:24.197748 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f98eec-af98-495d-b7a3-3692cdeceea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:24.198924 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f98eec-af98-495d-b7a3-3692cdeceea7 None admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:24.199755 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f98eec-af98-495d-b7a3-3692cdeceea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:24.200457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f98eec-af98-495d-b7a3-3692cdeceea7 None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4/users/9b90e57a93b44d24bc368ab2e96b81d2/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:24.202912 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f98eec-af98-495d-b7a3-3692cdeceea7 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d3731a0d0214a2ab2046a5d36d29aa4, user_id=9b90e57a93b44d24bc368ab2e96b81d2, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:24.205430 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f98eec-af98-495d-b7a3-3692cdeceea7 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:24.228697 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 73/136] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:55:24 2022] PUT /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4/users/9b90e57a93b44d24bc368ab2e96b81d2/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:55:24.230229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f98eec-af98-495d-b7a3-3692cdeceea7 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:24.274732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca19474b-afe4-4d3a-a17c-00d9b8841eec None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:24.385534 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c85fa-0f1a-41bf-8ff7-488a0b7c2756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:24.386200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c85fa-0f1a-41bf-8ff7-488a0b7c2756 None admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:24.386467 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c85fa-0f1a-41bf-8ff7-488a0b7c2756 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:24.386738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c85fa-0f1a-41bf-8ff7-488a0b7c2756 None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4/users/9b90e57a93b44d24bc368ab2e96b81d2/roles/9c6920b8e6f744a6a80371b4218605fd` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:24.388931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c85fa-0f1a-41bf-8ff7-488a0b7c2756 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d3731a0d0214a2ab2046a5d36d29aa4, user_id=9b90e57a93b44d24bc368ab2e96b81d2, role_id=9c6920b8e6f744a6a80371b4218605fd)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:24.390369 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c85fa-0f1a-41bf-8ff7-488a0b7c2756 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:24.409069 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 64/137] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:55:24 2022] PUT /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4/users/9b90e57a93b44d24bc368ab2e96b81d2/roles/9c6920b8e6f744a6a80371b4218605fd => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:55:24.409236 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f98eec-af98-495d-b7a3-3692cdeceea7 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:24.410292 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd93a86-e40d-4c06-bccc-ba05802c80ad None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:24.410883 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd93a86-e40d-4c06-bccc-ba05802c80ad None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:24.411190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd93a86-e40d-4c06-bccc-ba05802c80ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:24.429827 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:24.439378 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-7dd93a86-e40d-4c06-bccc-ba05802c80ad None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:24.440945 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7dd93a86-e40d-4c06-bccc-ba05802c80ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:24.455225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b1c60-f999-4c7f-920b-51d92da7f27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:24.455991 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b1c60-f999-4c7f-920b-51d92da7f27d None admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:24.456411 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b1c60-f999-4c7f-920b-51d92da7f27d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:24.456747 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b1c60-f999-4c7f-920b-51d92da7f27d None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4/users/95efeb6f9f5f497f9453dc5e9764070a/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:24.459004 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b1c60-f999-4c7f-920b-51d92da7f27d None admin] RBAC: Authorizing `identity:create_grant(project_id=2d3731a0d0214a2ab2046a5d36d29aa4, user_id=95efeb6f9f5f497f9453dc5e9764070a, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:24.460744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b1c60-f999-4c7f-920b-51d92da7f27d None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:24.480172 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:24.491616 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 74/138] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:55:24 2022] PUT /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4/users/95efeb6f9f5f497f9453dc5e9764070a/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:55:24.528206 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 65/139] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:24.819262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b1c60-f999-4c7f-920b-51d92da7f27d None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:24.939351 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b1c60-f999-4c7f-920b-51d92da7f27d None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:24.980006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d568f6a-209d-4cfc-847e-21a371fc60d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:24.980652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d568f6a-209d-4cfc-847e-21a371fc60d5 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:24.980933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d568f6a-209d-4cfc-847e-21a371fc60d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:24.981211 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d568f6a-209d-4cfc-847e-21a371fc60d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:24.982351 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d568f6a-209d-4cfc-847e-21a371fc60d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:24.983845 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d568f6a-209d-4cfc-847e-21a371fc60d5 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:24.985436 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 75/140] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:55:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:24.991102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd93a86-e40d-4c06-bccc-ba05802c80ad None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:25.149240 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd93a86-e40d-4c06-bccc-ba05802c80ad None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:25.198240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e6a38-8e31-4c98-a202-3baa3b853111 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:25.198568 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e6a38-8e31-4c98-a202-3baa3b853111 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:25.198797 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e6a38-8e31-4c98-a202-3baa3b853111 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:25.200206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 76/141] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:55:25.205249 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a5c40-8ee9-40a7-8ecc-b4212f8e376a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:25.206266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a5c40-8ee9-40a7-8ecc-b4212f8e376a None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:25.206747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a5c40-8ee9-40a7-8ecc-b4212f8e376a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:25.207304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a5c40-8ee9-40a7-8ecc-b4212f8e376a None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:25.208153 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a5c40-8ee9-40a7-8ecc-b4212f8e376a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:25.209785 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a5c40-8ee9-40a7-8ecc-b4212f8e376a None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:25.228299 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dbcf2-a50a-4eb5-b89f-9da8c08cae30 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:25.228586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dbcf2-a50a-4eb5-b89f-9da8c08cae30 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:25.229082 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dbcf2-a50a-4eb5-b89f-9da8c08cae30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:25.251540 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 66/142] 10.222.0.36 () {60 vars in 1132 bytes} [Fri Sep 2 08:55:24 2022] POST /identity/v3/users => generated 283 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:55:25.276233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-084dbcf2-a50a-4eb5-b89f-9da8c08cae30 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:25.277508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-084dbcf2-a50a-4eb5-b89f-9da8c08cae30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:25.340732 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:25.385818 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 77/143] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:25.879260 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a5c40-8ee9-40a7-8ecc-b4212f8e376a None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:25.908466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:25.991711 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a5c40-8ee9-40a7-8ecc-b4212f8e376a None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:26.000133 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e81c9-37d1-46c3-ab29-f878678701a6 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:26.001540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e81c9-37d1-46c3-ab29-f878678701a6 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:26.002025 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e81c9-37d1-46c3-ab29-f878678701a6 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:26.003965 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 78/144] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:26.015287 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4c646-937b-4cf8-982e-eecb7f2544ee None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:26.015858 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4c646-937b-4cf8-982e-eecb7f2544ee None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:26.016336 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c4c646-937b-4cf8-982e-eecb7f2544ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:26.043520 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11086458-d497-4fa2-b2cc-ebeb53681be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:26.044506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11086458-d497-4fa2-b2cc-ebeb53681be8 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:26.044986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11086458-d497-4fa2-b2cc-ebeb53681be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:26.045455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11086458-d497-4fa2-b2cc-ebeb53681be8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:26.046984 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-01c4c646-937b-4cf8-982e-eecb7f2544ee None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:26.048567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-01c4c646-937b-4cf8-982e-eecb7f2544ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:26.066322 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11086458-d497-4fa2-b2cc-ebeb53681be8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:26.073874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-11086458-d497-4fa2-b2cc-ebeb53681be8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 08:55:26.073874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:26.073874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:26.073874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:26.073874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:26.073874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:26.073874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:26.073874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:26.073874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:26.073874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:26.073874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:26.073874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:26.073874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:26.073874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:26.073874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:26.073874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:26.074824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:26.074824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:26.074824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:26.074824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:26.074824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:26.074824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:26.074824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:26.074824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:26.074824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:26.074824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:26.074824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:26.074824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:26.074824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:26.074824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:26.074824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:26.074824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:26.075626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:26.075626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:26.075626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:26.075626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:26.075626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:26.075626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:26.075626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:26.075626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:26.075626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:26.075626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:26.075626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 08:55:26.075626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:26.080678 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 67/145] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:26.091776 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11086458-d497-4fa2-b2cc-ebeb53681be8 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:26.207560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:26.219933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11086458-d497-4fa2-b2cc-ebeb53681be8 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:26.248564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 79/146] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:55:26.258233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0c49a-9c38-4cc8-a430-2877c1f1217e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:26.258864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0c49a-9c38-4cc8-a430-2877c1f1217e None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:26.259183 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0c49a-9c38-4cc8-a430-2877c1f1217e None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:26.259466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0c49a-9c38-4cc8-a430-2877c1f1217e None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:26.260150 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0c49a-9c38-4cc8-a430-2877c1f1217e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:26.260466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0c49a-9c38-4cc8-a430-2877c1f1217e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:26.261894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0c49a-9c38-4cc8-a430-2877c1f1217e None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:26.272157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 68/147] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:26.278176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4c646-937b-4cf8-982e-eecb7f2544ee None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:26.411114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c4c646-937b-4cf8-982e-eecb7f2544ee None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:26.413479 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:26.454082 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4e047-a6bf-4205-abf8-d7afb88ab887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:26.454727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4e047-a6bf-4205-abf8-d7afb88ab887 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:26.455006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4e047-a6bf-4205-abf8-d7afb88ab887 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:26.455301 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4e047-a6bf-4205-abf8-d7afb88ab887 None admin] PATH_INFO: `/v3/users/6ac5aac6d9b8480b9ec17074a26879ad` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:26.456446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4e047-a6bf-4205-abf8-d7afb88ab887 None admin] RBAC: Authorizing `identity:get_user(user_id=6ac5aac6d9b8480b9ec17074a26879ad)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:26.458138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4e047-a6bf-4205-abf8-d7afb88ab887 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:26.459800 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 80/148] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:26 2022] GET /identity/v3/users/6ac5aac6d9b8480b9ec17074a26879ad => generated 289 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:26.466224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0c49a-9c38-4cc8-a430-2877c1f1217e None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:26.586559 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0c49a-9c38-4cc8-a430-2877c1f1217e None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:26.622091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d931e2e-9b07-410e-9e49-e162f78ce164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:26.622728 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d931e2e-9b07-410e-9e49-e162f78ce164 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:26.623008 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d931e2e-9b07-410e-9e49-e162f78ce164 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:26.623310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d931e2e-9b07-410e-9e49-e162f78ce164 None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:26.624449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d931e2e-9b07-410e-9e49-e162f78ce164 None admin] RBAC: Authorizing `identity:get_project(project_id=2d3731a0d0214a2ab2046a5d36d29aa4)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:26.625897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d931e2e-9b07-410e-9e49-e162f78ce164 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:26.627863 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 69/149] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:26 2022] GET /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:26.634280 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4e047-a6bf-4205-abf8-d7afb88ab887 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:26.700916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d931e2e-9b07-410e-9e49-e162f78ce164 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:26.727538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:26.819387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4e047-a6bf-4205-abf8-d7afb88ab887 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:26.900431 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90101ad5-7839-4cbb-bdac-7e1ac76db2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:26.901077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90101ad5-7839-4cbb-bdac-7e1ac76db2ec None admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:26.901353 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90101ad5-7839-4cbb-bdac-7e1ac76db2ec None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:26.901621 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90101ad5-7839-4cbb-bdac-7e1ac76db2ec None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4/users/6ac5aac6d9b8480b9ec17074a26879ad/roles/b1bd3fa77e3d4c83aa57374cb35fd21f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:26.908343 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90101ad5-7839-4cbb-bdac-7e1ac76db2ec None admin] RBAC: Authorizing `identity:create_grant(project_id=2d3731a0d0214a2ab2046a5d36d29aa4, user_id=6ac5aac6d9b8480b9ec17074a26879ad, role_id=b1bd3fa77e3d4c83aa57374cb35fd21f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:26.909802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90101ad5-7839-4cbb-bdac-7e1ac76db2ec None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:26.913822 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d931e2e-9b07-410e-9e49-e162f78ce164 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:26.950491 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 81/150] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:55:26 2022] PUT /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4/users/6ac5aac6d9b8480b9ec17074a26879ad/roles/b1bd3fa77e3d4c83aa57374cb35fd21f => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:55:26.978373 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba5a78-758f-4b34-9aee-897a209e4d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:26.979415 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba5a78-758f-4b34-9aee-897a209e4d44 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:26.979911 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba5a78-758f-4b34-9aee-897a209e4d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:26.980376 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba5a78-758f-4b34-9aee-897a209e4d44 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:27.003782 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ba5a78-758f-4b34-9aee-897a209e4d44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:27.009651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-79ba5a78-758f-4b34-9aee-897a209e4d44 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:27.009651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:27.009651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:27.009651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:27.009651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:27.009651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:27.009651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:27.009651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:27.009651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:27.009651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:27.009651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:27.009651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:27.009651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:27.009651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:27.009651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:27.009651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:27.010536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:27.010536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:27.010536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:27.010536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:27.010536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:27.010536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:27.010536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:27.010536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:27.010536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:27.010536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:27.010536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:27.010536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:27.010536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:27.010536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:27.010536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:27.010536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:27.011331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:27.011331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:27.011331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:27.011331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:27.011331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:27.011331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:27.011331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:27.011331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:27.011331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:27.011331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:27.011331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:27.011331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:27.014005 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 70/151] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:55:27.020093 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90101ad5-7839-4cbb-bdac-7e1ac76db2ec None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:27.093927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:27.187933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90101ad5-7839-4cbb-bdac-7e1ac76db2ec None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:27.233071 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c154b2-1007-4daa-81ba-d59f447d0bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:27.233601 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c154b2-1007-4daa-81ba-d59f447d0bfd None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:27.233873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c154b2-1007-4daa-81ba-d59f447d0bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:27.234141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c154b2-1007-4daa-81ba-d59f447d0bfd None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:27.234808 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c154b2-1007-4daa-81ba-d59f447d0bfd None admin] RBAC: Adding query filter params (name=member) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:27.235126 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c154b2-1007-4daa-81ba-d59f447d0bfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:27.236511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c154b2-1007-4daa-81ba-d59f447d0bfd None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:27.250855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 82/152] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:27.256246 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba5a78-758f-4b34-9aee-897a209e4d44 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:27.383807 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:27.419438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba5a78-758f-4b34-9aee-897a209e4d44 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:27.461521 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd0f88-1115-4ee2-b86c-ca4ae7ceefe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:27.462173 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd0f88-1115-4ee2-b86c-ca4ae7ceefe7 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:27.462457 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd0f88-1115-4ee2-b86c-ca4ae7ceefe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:27.462734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd0f88-1115-4ee2-b86c-ca4ae7ceefe7 None admin] PATH_INFO: `/v3/users/9b90e57a93b44d24bc368ab2e96b81d2` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:27.466918 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd0f88-1115-4ee2-b86c-ca4ae7ceefe7 None admin] RBAC: Authorizing `identity:get_user(user_id=9b90e57a93b44d24bc368ab2e96b81d2)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:27.468461 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd0f88-1115-4ee2-b86c-ca4ae7ceefe7 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:27.470163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 71/153] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:27 2022] GET /identity/v3/users/9b90e57a93b44d24bc368ab2e96b81d2 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:27.480179 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c154b2-1007-4daa-81ba-d59f447d0bfd None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:27.560931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424e6ca-32df-4ca6-b738-f9bf9278c2e0 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:27.561237 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424e6ca-32df-4ca6-b738-f9bf9278c2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:27.561417 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b424e6ca-32df-4ca6-b738-f9bf9278c2e0 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:27.563093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 72/154] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:55:27.571416 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebeed1-6444-4f29-824f-0ac7afdfabd2 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:27.571699 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebeed1-6444-4f29-824f-0ac7afdfabd2 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:27.571976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebeed1-6444-4f29-824f-0ac7afdfabd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:27.602020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-b9ebeed1-6444-4f29-824f-0ac7afdfabd2 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:27.603261 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b9ebeed1-6444-4f29-824f-0ac7afdfabd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:27.631476 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:27.635651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c154b2-1007-4daa-81ba-d59f447d0bfd None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:27.696150 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6d9b4d-3a9f-4fb4-b824-3cdaf930bedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:27.696781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d9b4d-3a9f-4fb4-b824-3cdaf930bedd None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:27.697058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d9b4d-3a9f-4fb4-b824-3cdaf930bedd None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:27.697327 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d9b4d-3a9f-4fb4-b824-3cdaf930bedd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:27.756775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 73/155] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 02 08:55:27.758603 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf430c-5307-4fab-84ad-2db5d3443e90 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:27.758887 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf430c-5307-4fab-84ad-2db5d3443e90 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:27.759180 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adf430c-5307-4fab-84ad-2db5d3443e90 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:27.760582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 74/156] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:27.761439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91bddb-a129-4d60-bd80-59fb3820db57 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:27.761710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91bddb-a129-4d60-bd80-59fb3820db57 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:27.761976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e91bddb-a129-4d60-bd80-59fb3820db57 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:27.768117 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 75/157] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:27.769053 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ed0aa-d216-4bff-b944-685ef871a7ba None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:27.769330 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ed0aa-d216-4bff-b944-685ef871a7ba None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:27.769600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ed0aa-d216-4bff-b944-685ef871a7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:27.797227 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d9b4d-3a9f-4fb4-b824-3cdaf930bedd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:27.798723 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6d9b4d-3a9f-4fb4-b824-3cdaf930bedd None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:27.816188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-975ed0aa-d216-4bff-b944-685ef871a7ba None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:27.817362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-975ed0aa-d216-4bff-b944-685ef871a7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:27.913521 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:27.926722 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 83/158] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:27.929176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce26db-56fb-490e-a0f6-d849df5c85a4 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:27.929502 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce26db-56fb-490e-a0f6-d849df5c85a4 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:27.929778 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce26db-56fb-490e-a0f6-d849df5c85a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:27.949523 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 76/159] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:27.950183 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ed0aa-d216-4bff-b944-685ef871a7ba None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:27.957145 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:27.967390 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-47ce26db-56fb-490e-a0f6-d849df5c85a4 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:27.968567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-47ce26db-56fb-490e-a0f6-d849df5c85a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:28.059967 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ed0aa-d216-4bff-b944-685ef871a7ba None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:28.076029 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 84/160] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:28.098644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0c570-4139-4fa6-9c1f-7175a59bf4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:28.099318 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0c570-4139-4fa6-9c1f-7175a59bf4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:28.099592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0c570-4139-4fa6-9c1f-7175a59bf4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:28.099858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0c570-4139-4fa6-9c1f-7175a59bf4a3 None admin] PATH_INFO: `/v3/projects/93bf8f67aa39492d99306ade723efc09` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:28.103760 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f0c570-4139-4fa6-9c1f-7175a59bf4a3 None admin] RBAC: Authorizing `identity:get_project(project_id=93bf8f67aa39492d99306ade723efc09)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:28.105221 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f0c570-4139-4fa6-9c1f-7175a59bf4a3 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:28.106869 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 77/161] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:27 2022] GET /identity/v3/projects/93bf8f67aa39492d99306ade723efc09 => generated 312 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:28.112964 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce26db-56fb-490e-a0f6-d849df5c85a4 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:28.250031 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce26db-56fb-490e-a0f6-d849df5c85a4 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:28.273823 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:28.290303 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8dbf9-ab46-4211-bc00-811d02af3835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:28.291106 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8dbf9-ab46-4211-bc00-811d02af3835 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:28.291628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8dbf9-ab46-4211-bc00-811d02af3835 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:28.291900 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c8dbf9-ab46-4211-bc00-811d02af3835 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:28.292628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8dbf9-ab46-4211-bc00-811d02af3835 None admin] RBAC: Adding query filter params (role.id=08c2a54a85ef4bb9892a5769ef8a5d95, scope.project.id=93bf8f67aa39492d99306ade723efc09, user.id=9b90e57a93b44d24bc368ab2e96b81d2) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:28.292929 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8dbf9-ab46-4211-bc00-811d02af3835 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:28.294369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c8dbf9-ab46-4211-bc00-811d02af3835 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:28.300505 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 85/162] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:55:28 2022] GET /identity/v3/role_assignments?user.id=9b90e57a93b44d24bc368ab2e96b81d2&scope.project.id=93bf8f67aa39492d99306ade723efc09&role.id=08c2a54a85ef4bb9892a5769ef8a5d95 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:28.488528 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0c570-4139-4fa6-9c1f-7175a59bf4a3 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:28.566387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8dbf9-ab46-4211-bc00-811d02af3835 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:28.575674 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:28.611551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0c570-4139-4fa6-9c1f-7175a59bf4a3 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:28.662636 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08775dfe-8aa0-4b45-b3be-bd23fd6c54d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:28.663305 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08775dfe-8aa0-4b45-b3be-bd23fd6c54d5 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:28.667471 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08775dfe-8aa0-4b45-b3be-bd23fd6c54d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:28.667920 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08775dfe-8aa0-4b45-b3be-bd23fd6c54d5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:28.679174 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08775dfe-8aa0-4b45-b3be-bd23fd6c54d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:28.688495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-08775dfe-8aa0-4b45-b3be-bd23fd6c54d5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:55:28.688495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:28.688495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:28.688495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:28.688495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:28.688495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:28.688495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:28.688495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:28.688495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:28.688495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:28.688495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:28.688495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:28.688495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:28.688495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:28.688495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:28.688495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:28.689612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:28.689612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:28.689612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:28.689612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:28.689612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:28.689612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:28.689612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:28.689612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:28.689612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:28.689612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:28.689612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:28.689612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:28.689612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:28.689612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:28.689612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:28.689612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:28.690555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:28.690555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:28.690555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:28.690555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:28.690555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:28.690555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:28.690555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:28.690555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:28.690555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:28.690555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:28.690555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:55:28.690555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:28.692864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 78/163] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:55:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:28.700059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08775dfe-8aa0-4b45-b3be-bd23fd6c54d5 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:28.724778 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c8dbf9-ab46-4211-bc00-811d02af3835 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:28.780714 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba491ad-0c99-4713-952d-8230465668c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:28.781596 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba491ad-0c99-4713-952d-8230465668c6 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:28.782014 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba491ad-0c99-4713-952d-8230465668c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:28.782446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba491ad-0c99-4713-952d-8230465668c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:28.800152 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba491ad-0c99-4713-952d-8230465668c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:28.811107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-bba491ad-0c99-4713-952d-8230465668c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:55:28.811107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:28.811107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:28.811107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:28.811107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:28.811107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:28.811107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:28.811107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:28.811107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:28.811107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:28.811107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:28.811107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:28.811107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:28.811107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:28.811107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:28.811107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:28.812322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:28.812322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:28.812322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:28.812322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:28.812322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:28.812322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:28.812322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:28.812322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:28.812322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:28.812322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:28.812322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:28.812322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:28.812322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:28.812322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:28.812322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:28.812322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:28.813087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:28.813087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:28.813087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:28.813087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:28.813087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:28.813087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:28.813087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:28.813087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:28.813087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:28.813087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:28.813087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:55:28.813087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:28.815548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 86/164] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:55:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:28.816604 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba491ad-0c99-4713-952d-8230465668c6 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:28.889572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08775dfe-8aa0-4b45-b3be-bd23fd6c54d5 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:28.919454 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:28.927475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:28.934403 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5cade-14fd-494a-92dc-a97c43dde575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:28.935021 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5cade-14fd-494a-92dc-a97c43dde575 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:28.935339 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5cade-14fd-494a-92dc-a97c43dde575 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:28.935621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5cade-14fd-494a-92dc-a97c43dde575 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:28.936296 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5cade-14fd-494a-92dc-a97c43dde575 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:28.936905 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5cade-14fd-494a-92dc-a97c43dde575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:28.938279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5cade-14fd-494a-92dc-a97c43dde575 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:28.955755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 79/165] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:55:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:28.956620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5cade-14fd-494a-92dc-a97c43dde575 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:29.033277 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba491ad-0c99-4713-952d-8230465668c6 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:29.115789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca7226-37c5-479b-9584-f15e59c32f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:29.116599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca7226-37c5-479b-9584-f15e59c32f00 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:29.117001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca7226-37c5-479b-9584-f15e59c32f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:29.123093 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca7226-37c5-479b-9584-f15e59c32f00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:29.148584 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeca7226-37c5-479b-9584-f15e59c32f00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:29.155118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-eeca7226-37c5-479b-9584-f15e59c32f00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:29.155118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:29.155118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:29.155118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:29.155118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:29.155118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:29.155118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:29.155118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:29.155118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:29.155118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:29.155118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:29.155118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:29.155118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:29.155118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:29.155118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:29.155118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:29.156041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:29.156041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:29.156041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:29.156041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:29.156041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:29.156041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:29.156041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:29.156041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:29.156041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:29.156041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:29.156041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:29.156041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:29.156041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:29.156041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:29.156041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:29.156041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:29.156840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:29.156840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:29.156840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:29.156840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:29.156840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:29.156840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:29.156840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:29.156840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:29.156840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:29.156840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:29.156840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:29.156840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:29.156840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 87/166] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:29.167993 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca7226-37c5-479b-9584-f15e59c32f00 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:29.171674 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5cade-14fd-494a-92dc-a97c43dde575 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:29.252341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb27258-9501-45fc-92cd-3b49d6c5f329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:29.253415 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb27258-9501-45fc-92cd-3b49d6c5f329 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:29.253907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb27258-9501-45fc-92cd-3b49d6c5f329 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:29.254389 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb27258-9501-45fc-92cd-3b49d6c5f329 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:29.255293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb27258-9501-45fc-92cd-3b49d6c5f329 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:29.255822 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb27258-9501-45fc-92cd-3b49d6c5f329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:29.257921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb27258-9501-45fc-92cd-3b49d6c5f329 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:29.274677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 80/167] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:29.275785 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb27258-9501-45fc-92cd-3b49d6c5f329 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:29.408761 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:29.433214 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca7226-37c5-479b-9584-f15e59c32f00 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:29.451374 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:29.511425 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb27258-9501-45fc-92cd-3b49d6c5f329 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:29.525200 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860057f5-bf1b-43bd-bb1b-2e426932f40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:29.525839 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860057f5-bf1b-43bd-bb1b-2e426932f40a None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:29.526115 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860057f5-bf1b-43bd-bb1b-2e426932f40a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:29.526384 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860057f5-bf1b-43bd-bb1b-2e426932f40a None admin] PATH_INFO: `/v3/users/9b90e57a93b44d24bc368ab2e96b81d2` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:29.531884 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860057f5-bf1b-43bd-bb1b-2e426932f40a None admin] RBAC: Authorizing `identity:get_user(user_id=9b90e57a93b44d24bc368ab2e96b81d2)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:29.533357 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860057f5-bf1b-43bd-bb1b-2e426932f40a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:29.534934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 88/168] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:29 2022] GET /identity/v3/users/9b90e57a93b44d24bc368ab2e96b81d2 => generated 275 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:29.535769 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860057f5-bf1b-43bd-bb1b-2e426932f40a None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:29.593263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652be42a-c00e-4198-a715-fb3052d9b965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:29.593894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652be42a-c00e-4198-a715-fb3052d9b965 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:29.594174 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652be42a-c00e-4198-a715-fb3052d9b965 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:29.594456 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652be42a-c00e-4198-a715-fb3052d9b965 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:29.595138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652be42a-c00e-4198-a715-fb3052d9b965 None admin] RBAC: Adding query filter params (name=member) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:29.595447 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652be42a-c00e-4198-a715-fb3052d9b965 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:29.596843 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652be42a-c00e-4198-a715-fb3052d9b965 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:29.613956 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 81/169] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:29.614639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652be42a-c00e-4198-a715-fb3052d9b965 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:29.726337 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860057f5-bf1b-43bd-bb1b-2e426932f40a None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:29.728826 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652be42a-c00e-4198-a715-fb3052d9b965 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:29.746356 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:29.774237 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89433715-0f22-4654-baa3-e1175cc8edce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:29.775939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89433715-0f22-4654-baa3-e1175cc8edce None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:29.776817 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89433715-0f22-4654-baa3-e1175cc8edce None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:29.777755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89433715-0f22-4654-baa3-e1175cc8edce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:29.783538 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cd233a-3cc0-4dc7-b8ae-4d20a35be97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:29.784420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd233a-3cc0-4dc7-b8ae-4d20a35be97e None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:29.784898 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd233a-3cc0-4dc7-b8ae-4d20a35be97e None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:29.785445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cd233a-3cc0-4dc7-b8ae-4d20a35be97e None admin] PATH_INFO: `/v3/users/95efeb6f9f5f497f9453dc5e9764070a` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:29.791605 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd233a-3cc0-4dc7-b8ae-4d20a35be97e None admin] RBAC: Authorizing `identity:get_user(user_id=95efeb6f9f5f497f9453dc5e9764070a)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:29.793419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cd233a-3cc0-4dc7-b8ae-4d20a35be97e None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:29.795550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 89/170] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:29 2022] GET /identity/v3/users/95efeb6f9f5f497f9453dc5e9764070a => generated 247 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:29.796933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08eeca-91dd-48e7-97b7-898cd8b4355f None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:29.799373 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08eeca-91dd-48e7-97b7-898cd8b4355f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:29.799871 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08eeca-91dd-48e7-97b7-898cd8b4355f None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:29.801359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 90/171] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:29.803481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08eeca-91dd-48e7-97b7-898cd8b4355f None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:29.892684 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89433715-0f22-4654-baa3-e1175cc8edce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:29.894591 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89433715-0f22-4654-baa3-e1175cc8edce None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:29.906506 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:30.008488 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08eeca-91dd-48e7-97b7-898cd8b4355f None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:30.058087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607069b-7887-4ad3-b7bd-5ac241b8f774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:30.059131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607069b-7887-4ad3-b7bd-5ac241b8f774 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:30.060596 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607069b-7887-4ad3-b7bd-5ac241b8f774 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:30.062725 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5607069b-7887-4ad3-b7bd-5ac241b8f774 None admin] PATH_INFO: `/v3/users/9b90e57a93b44d24bc368ab2e96b81d2` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:30.065612 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5607069b-7887-4ad3-b7bd-5ac241b8f774 None admin] RBAC: Authorizing `identity:get_user(user_id=9b90e57a93b44d24bc368ab2e96b81d2)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:30.068625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5607069b-7887-4ad3-b7bd-5ac241b8f774 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:30.074038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 91/172] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:29 2022] GET /identity/v3/users/9b90e57a93b44d24bc368ab2e96b81d2 => generated 275 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:30.075227 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607069b-7887-4ad3-b7bd-5ac241b8f774 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:30.098358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 82/173] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:30.100299 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc79b1-72d7-4bce-ac97-c68fb3d4b2ad None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:30.101998 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc79b1-72d7-4bce-ac97-c68fb3d4b2ad None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:30.102470 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc79b1-72d7-4bce-ac97-c68fb3d4b2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:30.124522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-3bdc79b1-72d7-4bce-ac97-c68fb3d4b2ad None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:30.126051 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-3bdc79b1-72d7-4bce-ac97-c68fb3d4b2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:30.138273 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:30.195604 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 83/174] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:30.196290 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc79b1-72d7-4bce-ac97-c68fb3d4b2ad None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:30.227812 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5607069b-7887-4ad3-b7bd-5ac241b8f774 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:30.242883 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:30.266766 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2727fb-179d-40b8-86ea-df5227cf6bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:30.267339 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2727fb-179d-40b8-86ea-df5227cf6bae None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:30.267623 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2727fb-179d-40b8-86ea-df5227cf6bae None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:30.267881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2727fb-179d-40b8-86ea-df5227cf6bae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:30.313059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc79b1-72d7-4bce-ac97-c68fb3d4b2ad None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:30.341892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2727fb-179d-40b8-86ea-df5227cf6bae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:30.344094 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2727fb-179d-40b8-86ea-df5227cf6bae None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:30.355814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e21e9a-36d8-4146-94f2-23087d018cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:30.356444 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e21e9a-36d8-4146-94f2-23087d018cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:30.356705 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e21e9a-36d8-4146-94f2-23087d018cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:30.356978 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e21e9a-36d8-4146-94f2-23087d018cd7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:30.373549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:30.428564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e21e9a-36d8-4146-94f2-23087d018cd7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:30.430048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e21e9a-36d8-4146-94f2-23087d018cd7 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:30.459415 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 92/175] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:30.459415 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2727fb-179d-40b8-86ea-df5227cf6bae None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:30.474227 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:30.528139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 84/176] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:30.529297 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3833c-43b1-40b3-8657-2fd33e4aec62 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:30.529573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3833c-43b1-40b3-8657-2fd33e4aec62 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:30.529841 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca3833c-43b1-40b3-8657-2fd33e4aec62 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:30.531118 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 85/177] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:30.532167 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e69df-762e-4b39-9915-d5e1403ab156 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:30.532427 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e69df-762e-4b39-9915-d5e1403ab156 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:30.532698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e69df-762e-4b39-9915-d5e1403ab156 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:30.533865 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 86/178] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:30.534851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677fb9c4-e889-433c-b12f-50f5ce1e3db5 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:30.535321 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677fb9c4-e889-433c-b12f-50f5ce1e3db5 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:30.535588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677fb9c4-e889-433c-b12f-50f5ce1e3db5 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:30.537031 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 87/179] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:30.537714 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677fb9c4-e889-433c-b12f-50f5ce1e3db5 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:30.567686 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2727fb-179d-40b8-86ea-df5227cf6bae None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:30.583963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:30.602405 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead75c65-d1c3-44b9-abf4-a9b2a141fda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:30.603092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead75c65-d1c3-44b9-abf4-a9b2a141fda2 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:30.603366 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead75c65-d1c3-44b9-abf4-a9b2a141fda2 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:30.603639 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead75c65-d1c3-44b9-abf4-a9b2a141fda2 None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:30.604762 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead75c65-d1c3-44b9-abf4-a9b2a141fda2 None admin] RBAC: Authorizing `identity:get_project(project_id=2d3731a0d0214a2ab2046a5d36d29aa4)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:30.606220 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead75c65-d1c3-44b9-abf4-a9b2a141fda2 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:30.607872 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 93/180] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:30 2022] GET /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:30.608691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead75c65-d1c3-44b9-abf4-a9b2a141fda2 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:30.637940 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677fb9c4-e889-433c-b12f-50f5ce1e3db5 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:30.671851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9773f6-43f8-41d5-93e9-114b34165235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:30.672475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9773f6-43f8-41d5-93e9-114b34165235 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:30.672760 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9773f6-43f8-41d5-93e9-114b34165235 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:30.673031 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9773f6-43f8-41d5-93e9-114b34165235 None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:30.674150 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9773f6-43f8-41d5-93e9-114b34165235 None admin] RBAC: Authorizing `identity:get_project(project_id=2d3731a0d0214a2ab2046a5d36d29aa4)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:30.675630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9773f6-43f8-41d5-93e9-114b34165235 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:30.677255 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 88/181] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:30 2022] GET /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:30.678340 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbc1df-ee2d-457b-b2d5-20d8b3728fb1 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:30.678618 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbc1df-ee2d-457b-b2d5-20d8b3728fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:30.678898 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbc1df-ee2d-457b-b2d5-20d8b3728fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:30.690443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:30.696727 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-d9fbc1df-ee2d-457b-b2d5-20d8b3728fb1 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:30.697974 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-d9fbc1df-ee2d-457b-b2d5-20d8b3728fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:30.709822 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead75c65-d1c3-44b9-abf4-a9b2a141fda2 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:30.743326 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bc88b-bd6d-4e8e-8121-47734d1466f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:30.743956 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bc88b-bd6d-4e8e-8121-47734d1466f7 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:30.744233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bc88b-bd6d-4e8e-8121-47734d1466f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:30.744511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1bc88b-bd6d-4e8e-8121-47734d1466f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:30.754320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1bc88b-bd6d-4e8e-8121-47734d1466f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:30.759675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-1d1bc88b-bd6d-4e8e-8121-47734d1466f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:55:30.759675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:30.759675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:30.759675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:30.759675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:30.759675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:30.759675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:30.759675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:30.759675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:30.759675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:30.759675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:30.759675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:30.759675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:30.759675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:30.759675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:30.759675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:30.760785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:30.760785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:30.760785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:30.760785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:30.760785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:30.760785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:30.760785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:30.760785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:30.760785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:30.760785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:30.760785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:30.760785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:30.760785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:30.760785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:30.760785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:30.760785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:30.761628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:30.761628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:30.761628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:30.761628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:30.761628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:30.761628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:30.761628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:30.761628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:30.761628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:30.761628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:30.761628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:55:30.761628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:30.761628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 94/182] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:55:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:30.761628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bc88b-bd6d-4e8e-8121-47734d1466f7 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:30.764676 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 89/183] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:30.765828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfdaf3-ade3-4ae7-bc6c-5b609b622be1 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:30.766270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfdaf3-ade3-4ae7-bc6c-5b609b622be1 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:30.766429 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dfdaf3-ade3-4ae7-bc6c-5b609b622be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:30.784165 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-21dfdaf3-ade3-4ae7-bc6c-5b609b622be1 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:30.785271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-21dfdaf3-ade3-4ae7-bc6c-5b609b622be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:30.824776 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:30.859285 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 90/184] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:30.861307 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1835870-362f-4281-ba69-dd9c6d973056 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:30.861736 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1835870-362f-4281-ba69-dd9c6d973056 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:30.862106 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1835870-362f-4281-ba69-dd9c6d973056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:30.866141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1bc88b-bd6d-4e8e-8121-47734d1466f7 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:30.893279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-a1835870-362f-4281-ba69-dd9c6d973056 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:30.894616 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-a1835870-362f-4281-ba69-dd9c6d973056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:30.906502 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c5379-ecb1-4b4b-a65b-7ca682812fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:30.907477 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c5379-ecb1-4b4b-a65b-7ca682812fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:30.907947 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c5379-ecb1-4b4b-a65b-7ca682812fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:30.908431 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c5379-ecb1-4b4b-a65b-7ca682812fd2 None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:30.909713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c5379-ecb1-4b4b-a65b-7ca682812fd2 None admin] RBAC: Authorizing `identity:get_project(project_id=2d3731a0d0214a2ab2046a5d36d29aa4)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:30.911643 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4c5379-ecb1-4b4b-a65b-7ca682812fd2 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:30.913522 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 95/185] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:30 2022] GET /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:30.914626 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c5379-ecb1-4b4b-a65b-7ca682812fd2 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:30.925782 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:30.970020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 91/186] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:30.971355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1835870-362f-4281-ba69-dd9c6d973056 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:31.018540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c5379-ecb1-4b4b-a65b-7ca682812fd2 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:31.053477 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9267a28b-d3f0-417d-ae3d-b4ae543d2c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:31.054835 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267a28b-d3f0-417d-ae3d-b4ae543d2c1b None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:31.055320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267a28b-d3f0-417d-ae3d-b4ae543d2c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:31.055745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9267a28b-d3f0-417d-ae3d-b4ae543d2c1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:31.056605 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9267a28b-d3f0-417d-ae3d-b4ae543d2c1b None admin] RBAC: Adding query filter params (role.id=9c6920b8e6f744a6a80371b4218605fd, scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4, user.id=9b90e57a93b44d24bc368ab2e96b81d2) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:31.057103 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9267a28b-d3f0-417d-ae3d-b4ae543d2c1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:31.058698 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9267a28b-d3f0-417d-ae3d-b4ae543d2c1b None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:31.065235 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 96/187] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:55:30 2022] GET /identity/v3/role_assignments?user.id=9b90e57a93b44d24bc368ab2e96b81d2&scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4&role.id=9c6920b8e6f744a6a80371b4218605fd => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:31.066256 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9267a28b-d3f0-417d-ae3d-b4ae543d2c1b None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:31.077703 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1835870-362f-4281-ba69-dd9c6d973056 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:31.091487 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:31.120980 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb04a76-0e1a-4f14-a1fb-1b6ffabcb3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:31.121646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb04a76-0e1a-4f14-a1fb-1b6ffabcb3de None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:31.121935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb04a76-0e1a-4f14-a1fb-1b6ffabcb3de None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:31.122138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb04a76-0e1a-4f14-a1fb-1b6ffabcb3de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:31.122882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb04a76-0e1a-4f14-a1fb-1b6ffabcb3de None admin] RBAC: Adding query filter params (role.id=a0a76de72fba4036bc1e3e64df717c7f, scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4, user.id=95efeb6f9f5f497f9453dc5e9764070a) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:31.123214 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb04a76-0e1a-4f14-a1fb-1b6ffabcb3de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:31.124663 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb04a76-0e1a-4f14-a1fb-1b6ffabcb3de None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:31.133625 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 92/188] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:55:30 2022] GET /identity/v3/role_assignments?user.id=95efeb6f9f5f497f9453dc5e9764070a&scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4&role.id=a0a76de72fba4036bc1e3e64df717c7f => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:31.139654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb04a76-0e1a-4f14-a1fb-1b6ffabcb3de None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:31.197079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:31.287549 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9267a28b-d3f0-417d-ae3d-b4ae543d2c1b None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:31.351055 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0d7e6-80db-4ee6-9237-52d147e5abd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:31.352743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0d7e6-80db-4ee6-9237-52d147e5abd5 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:31.352743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0d7e6-80db-4ee6-9237-52d147e5abd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:31.352743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0d7e6-80db-4ee6-9237-52d147e5abd5 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:31.352985 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0d7e6-80db-4ee6-9237-52d147e5abd5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:31.353176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0d7e6-80db-4ee6-9237-52d147e5abd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:31.354529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0d7e6-80db-4ee6-9237-52d147e5abd5 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:31.356731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb04a76-0e1a-4f14-a1fb-1b6ffabcb3de None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:31.368604 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 97/189] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:31.369493 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0d7e6-80db-4ee6-9237-52d147e5abd5 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:31.398758 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b3b12-e274-4d09-a1f2-c1e2abfcee7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:31.399397 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b3b12-e274-4d09-a1f2-c1e2abfcee7c None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:31.399697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b3b12-e274-4d09-a1f2-c1e2abfcee7c None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:31.399964 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b3b12-e274-4d09-a1f2-c1e2abfcee7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:31.400690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b3b12-e274-4d09-a1f2-c1e2abfcee7c None admin] RBAC: Adding query filter params (role.id=08c2a54a85ef4bb9892a5769ef8a5d95, scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4, user.id=9b90e57a93b44d24bc368ab2e96b81d2) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:31.400990 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b3b12-e274-4d09-a1f2-c1e2abfcee7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:31.402420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b3b12-e274-4d09-a1f2-c1e2abfcee7c None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:31.408672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 93/190] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:55:31 2022] GET /identity/v3/role_assignments?user.id=9b90e57a93b44d24bc368ab2e96b81d2&scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4&role.id=08c2a54a85ef4bb9892a5769ef8a5d95 => generated 608 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:31.409402 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b3b12-e274-4d09-a1f2-c1e2abfcee7c None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:31.451021 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:31.498093 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0d7e6-80db-4ee6-9237-52d147e5abd5 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:31.534696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b3b12-e274-4d09-a1f2-c1e2abfcee7c None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:31.540751 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9d3d3-21c2-4d8e-a028-4175f8c1e3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:31.541415 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9d3d3-21c2-4d8e-a028-4175f8c1e3ed None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:31.541682 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9d3d3-21c2-4d8e-a028-4175f8c1e3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:31.541950 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b9d3d3-21c2-4d8e-a028-4175f8c1e3ed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:31.544633 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:31.560030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b9d3d3-21c2-4d8e-a028-4175f8c1e3ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:31.568987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-34b9d3d3-21c2-4d8e-a028-4175f8c1e3ed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 08:55:31.568987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:31.568987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:31.568987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:31.568987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:31.568987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:31.568987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:31.568987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:31.568987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:31.568987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:31.568987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:31.568987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:31.568987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:31.568987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:31.568987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:31.568987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:31.569992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:31.569992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:31.569992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:31.569992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:31.569992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:31.569992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:31.569992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:31.569992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:31.569992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:31.569992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:31.569992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:31.569992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:31.569992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:31.569992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:31.569992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:31.569992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:31.570807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:31.570807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:31.570807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:31.570807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:31.570807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:31.570807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:31.570807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:31.570807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:31.570807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:31.570807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:31.570807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 08:55:31.570807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:31.570807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 98/191] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:31.571527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9d3d3-21c2-4d8e-a028-4175f8c1e3ed None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:31.578875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37ad60-f525-4bc4-9e76-88ff69fab194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:31.579558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37ad60-f525-4bc4-9e76-88ff69fab194 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:31.579841 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37ad60-f525-4bc4-9e76-88ff69fab194 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:31.580114 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37ad60-f525-4bc4-9e76-88ff69fab194 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:31.581241 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37ad60-f525-4bc4-9e76-88ff69fab194 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:31.582703 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da37ad60-f525-4bc4-9e76-88ff69fab194 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:31.584403 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 94/192] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:55:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:31.585169 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37ad60-f525-4bc4-9e76-88ff69fab194 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:31.683822 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b9d3d3-21c2-4d8e-a028-4175f8c1e3ed None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:31.688512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:31.691713 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da37ad60-f525-4bc4-9e76-88ff69fab194 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:31.717016 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce290e5-b7b9-40c9-a22b-9f2bf6c5ed5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:31.717638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce290e5-b7b9-40c9-a22b-9f2bf6c5ed5e None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:31.717914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce290e5-b7b9-40c9-a22b-9f2bf6c5ed5e None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:31.718187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce290e5-b7b9-40c9-a22b-9f2bf6c5ed5e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:31.725356 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e938c-ea53-429a-8175-3240cdcd8b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:31.726677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e938c-ea53-429a-8175-3240cdcd8b0f None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:31.726954 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e938c-ea53-429a-8175-3240cdcd8b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:31.727267 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136e938c-ea53-429a-8175-3240cdcd8b0f None admin] PATH_INFO: `/v3/users/ea62c6c957364b18945d11e0fd283e69` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:31.728988 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce290e5-b7b9-40c9-a22b-9f2bf6c5ed5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:31.733724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-2ce290e5-b7b9-40c9-a22b-9f2bf6c5ed5e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:55:31.733724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:31.733724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:31.733724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:31.733724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:31.733724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:31.733724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:31.733724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:31.733724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:31.733724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:31.733724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:31.733724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:31.733724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:31.733724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:31.733724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:31.733724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:31.734770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:31.734770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:31.734770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:31.734770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:31.734770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:31.734770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:31.734770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:31.734770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:31.734770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:31.734770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:31.734770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:31.734770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:31.734770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:31.734770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:31.734770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:31.734770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:31.735779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:31.735779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:31.735779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:31.735779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:31.735779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:31.735779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:31.735779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:31.735779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:31.735779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:31.735779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:31.735779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:55:31.735779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:31.735779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 99/193] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:55:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:31.735779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce290e5-b7b9-40c9-a22b-9f2bf6c5ed5e None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:31.742413 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e938c-ea53-429a-8175-3240cdcd8b0f None admin] RBAC: Authorizing `identity:get_user(user_id=ea62c6c957364b18945d11e0fd283e69)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:31.743888 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136e938c-ea53-429a-8175-3240cdcd8b0f None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:31.745533 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 95/194] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:31 2022] GET /identity/v3/users/ea62c6c957364b18945d11e0fd283e69 => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:31.750794 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e938c-ea53-429a-8175-3240cdcd8b0f None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:31.789652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:31.833940 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce290e5-b7b9-40c9-a22b-9f2bf6c5ed5e None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:31.849519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136e938c-ea53-429a-8175-3240cdcd8b0f None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:31.868743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acffffb8-246c-4938-94c8-8f75f08a2756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:31.869345 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffffb8-246c-4938-94c8-8f75f08a2756 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:31.869617 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffffb8-246c-4938-94c8-8f75f08a2756 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:31.869885 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acffffb8-246c-4938-94c8-8f75f08a2756 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:31.870532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acffffb8-246c-4938-94c8-8f75f08a2756 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:31.870824 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acffffb8-246c-4938-94c8-8f75f08a2756 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:31.872258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acffffb8-246c-4938-94c8-8f75f08a2756 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:31.880351 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 100/195] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:31.880351 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acffffb8-246c-4938-94c8-8f75f08a2756 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:31.887253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3795b70-baa6-4c39-9392-b9b5953acdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:31.887878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3795b70-baa6-4c39-9392-b9b5953acdaf None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:31.888154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3795b70-baa6-4c39-9392-b9b5953acdaf None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:31.888423 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3795b70-baa6-4c39-9392-b9b5953acdaf None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:31.889083 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3795b70-baa6-4c39-9392-b9b5953acdaf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:31.890518 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3795b70-baa6-4c39-9392-b9b5953acdaf None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:31.911940 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 96/196] 10.222.0.36 () {60 vars in 1132 bytes} [Fri Sep 2 08:55:31 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:55:31.912748 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3795b70-baa6-4c39-9392-b9b5953acdaf None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:31.944913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:32.005901 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acffffb8-246c-4938-94c8-8f75f08a2756 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:32.025740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:32.045110 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3795b70-baa6-4c39-9392-b9b5953acdaf None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:32.045640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc3a9b-0143-430d-b99e-5a1f6df19724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:32.046317 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc3a9b-0143-430d-b99e-5a1f6df19724 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:32.046656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc3a9b-0143-430d-b99e-5a1f6df19724 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:32.046991 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc3a9b-0143-430d-b99e-5a1f6df19724 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:32.048088 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc3a9b-0143-430d-b99e-5a1f6df19724 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:32.048088 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc3a9b-0143-430d-b99e-5a1f6df19724 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:32.049409 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc3a9b-0143-430d-b99e-5a1f6df19724 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:32.058694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 101/197] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:55:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:32.059686 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc3a9b-0143-430d-b99e-5a1f6df19724 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:32.089181 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6883ea-0b6d-4248-98aa-9c95fc0a4c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:32.089796 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6883ea-0b6d-4248-98aa-9c95fc0a4c20 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:32.090067 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6883ea-0b6d-4248-98aa-9c95fc0a4c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:32.090335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6883ea-0b6d-4248-98aa-9c95fc0a4c20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:32.168988 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc3a9b-0143-430d-b99e-5a1f6df19724 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:32.177104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6883ea-0b6d-4248-98aa-9c95fc0a4c20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:32.180467 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6883ea-0b6d-4248-98aa-9c95fc0a4c20 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:32.200560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:32.211091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8eb3ae-5faa-4b1f-bf71-ed4f08c1411c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:32.211874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8eb3ae-5faa-4b1f-bf71-ed4f08c1411c None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:32.212253 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8eb3ae-5faa-4b1f-bf71-ed4f08c1411c None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:32.212619 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8eb3ae-5faa-4b1f-bf71-ed4f08c1411c None admin] PATH_INFO: `/v3/users/6ac5aac6d9b8480b9ec17074a26879ad` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:32.214538 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8eb3ae-5faa-4b1f-bf71-ed4f08c1411c None admin] RBAC: Authorizing `identity:get_user(user_id=6ac5aac6d9b8480b9ec17074a26879ad)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:32.216830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8eb3ae-5faa-4b1f-bf71-ed4f08c1411c None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:32.218424 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 102/198] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:32 2022] GET /identity/v3/users/6ac5aac6d9b8480b9ec17074a26879ad => generated 289 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:32.219201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8eb3ae-5faa-4b1f-bf71-ed4f08c1411c None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:32.283657 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:32.324144 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 97/199] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:32.325149 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6883ea-0b6d-4248-98aa-9c95fc0a4c20 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:32.352465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8eb3ae-5faa-4b1f-bf71-ed4f08c1411c None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:32.389459 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2042e7-eecb-41f8-942e-f8d8dfdb63ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:32.390095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2042e7-eecb-41f8-942e-f8d8dfdb63ac None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:32.390372 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2042e7-eecb-41f8-942e-f8d8dfdb63ac None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:32.390643 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2042e7-eecb-41f8-942e-f8d8dfdb63ac None admin] PATH_INFO: `/v3/users/ea62c6c957364b18945d11e0fd283e69` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:32.391892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2042e7-eecb-41f8-942e-f8d8dfdb63ac None admin] RBAC: Authorizing `identity:get_user(user_id=ea62c6c957364b18945d11e0fd283e69)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:32.393445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2042e7-eecb-41f8-942e-f8d8dfdb63ac None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:32.395014 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 103/200] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:32 2022] GET /identity/v3/users/ea62c6c957364b18945d11e0fd283e69 => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:32.395832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2042e7-eecb-41f8-942e-f8d8dfdb63ac None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:32.446918 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6883ea-0b6d-4248-98aa-9c95fc0a4c20 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:32.462842 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:32.485144 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a643c0-4ace-4b6d-9b6a-e043517e0de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:32.485757 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a643c0-4ace-4b6d-9b6a-e043517e0de1 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:32.486155 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a643c0-4ace-4b6d-9b6a-e043517e0de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:32.486524 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a643c0-4ace-4b6d-9b6a-e043517e0de1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:32.513041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2042e7-eecb-41f8-942e-f8d8dfdb63ac None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:32.535440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:32.552115 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb36d6-4014-4840-a651-467028e8a23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:32.552989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb36d6-4014-4840-a651-467028e8a23d None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:32.553420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb36d6-4014-4840-a651-467028e8a23d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:32.553897 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb36d6-4014-4840-a651-467028e8a23d None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:32.555918 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb36d6-4014-4840-a651-467028e8a23d None admin] RBAC: Authorizing `identity:get_project(project_id=78733ddb1b714faf83957113a15c51bc)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:32.557779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb36d6-4014-4840-a651-467028e8a23d None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:32.559237 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a643c0-4ace-4b6d-9b6a-e043517e0de1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:32.559770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 104/201] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:32 2022] GET /identity/v3/projects/78733ddb1b714faf83957113a15c51bc => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:32.560677 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb36d6-4014-4840-a651-467028e8a23d None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:32.561097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a643c0-4ace-4b6d-9b6a-e043517e0de1 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:32.667905 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 98/202] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:32.668760 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a643c0-4ace-4b6d-9b6a-e043517e0de1 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:32.684235 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:32.695843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb36d6-4014-4840-a651-467028e8a23d None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:32.732772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff500d-e154-4b35-9243-84e5aedb1853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:32.733348 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff500d-e154-4b35-9243-84e5aedb1853 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:32.733608 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff500d-e154-4b35-9243-84e5aedb1853 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:32.733876 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff500d-e154-4b35-9243-84e5aedb1853 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:32.785195 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:32.804512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a643c0-4ace-4b6d-9b6a-e043517e0de1 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:32.805924 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff500d-e154-4b35-9243-84e5aedb1853 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:32.807428 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aff500d-e154-4b35-9243-84e5aedb1853 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:32.851368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e54cbf-8dfe-42bc-b72c-96857165d1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:32.852528 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e54cbf-8dfe-42bc-b72c-96857165d1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:32.853011 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e54cbf-8dfe-42bc-b72c-96857165d1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:32.853682 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e54cbf-8dfe-42bc-b72c-96857165d1f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:32.855055 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e54cbf-8dfe-42bc-b72c-96857165d1f3 None admin] RBAC: Adding query filter params (role.id=a0a76de72fba4036bc1e3e64df717c7f, scope.project.id=78733ddb1b714faf83957113a15c51bc, user.id=ea62c6c957364b18945d11e0fd283e69) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:32.855836 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e54cbf-8dfe-42bc-b72c-96857165d1f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:32.858130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e54cbf-8dfe-42bc-b72c-96857165d1f3 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:32.865554 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 99/203] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:55:32 2022] GET /identity/v3/role_assignments?user.id=ea62c6c957364b18945d11e0fd283e69&scope.project.id=78733ddb1b714faf83957113a15c51bc&role.id=a0a76de72fba4036bc1e3e64df717c7f => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:32.866793 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e54cbf-8dfe-42bc-b72c-96857165d1f3 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:32.917488 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 105/204] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:32.923691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff500d-e154-4b35-9243-84e5aedb1853 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:32.972110 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:32.996637 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e54cbf-8dfe-42bc-b72c-96857165d1f3 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:33.013071 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:33.034995 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff500d-e154-4b35-9243-84e5aedb1853 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:33.035608 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf95fe-af20-457a-842b-9874a631fa35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:33.036427 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf95fe-af20-457a-842b-9874a631fa35 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:33.036805 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf95fe-af20-457a-842b-9874a631fa35 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:33.037181 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf95fe-af20-457a-842b-9874a631fa35 None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:33.039352 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf95fe-af20-457a-842b-9874a631fa35 None admin] RBAC: Authorizing `identity:get_project(project_id=2d3731a0d0214a2ab2046a5d36d29aa4)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:33.040284 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf95fe-af20-457a-842b-9874a631fa35 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:33.042056 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 100/205] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:32 2022] GET /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:33.048262 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf95fe-af20-457a-842b-9874a631fa35 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:33.074903 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a8edc0-1fd2-4489-9d10-9badd0a82cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:33.075816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8edc0-1fd2-4489-9d10-9badd0a82cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:33.076258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8edc0-1fd2-4489-9d10-9badd0a82cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:33.076664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8edc0-1fd2-4489-9d10-9badd0a82cc8 None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:33.077984 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a8edc0-1fd2-4489-9d10-9badd0a82cc8 None admin] RBAC: Authorizing `identity:get_project(project_id=78733ddb1b714faf83957113a15c51bc)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:33.079698 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a8edc0-1fd2-4489-9d10-9badd0a82cc8 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:33.087272 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 106/206] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:32 2022] GET /identity/v3/projects/78733ddb1b714faf83957113a15c51bc => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:33.088886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a8edc0-1fd2-4489-9d10-9badd0a82cc8 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:33.180275 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf95fe-af20-457a-842b-9874a631fa35 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:33.202185 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a8edc0-1fd2-4489-9d10-9badd0a82cc8 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:33.219428 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39d757-f865-4317-b72b-9e4ca7f4af4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:33.220120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39d757-f865-4317-b72b-9e4ca7f4af4d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:33.220478 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39d757-f865-4317-b72b-9e4ca7f4af4d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:33.220837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39d757-f865-4317-b72b-9e4ca7f4af4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:33.221690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39d757-f865-4317-b72b-9e4ca7f4af4d None admin] RBAC: Adding query filter params (role.id=b1bd3fa77e3d4c83aa57374cb35fd21f, scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4, user.id=6ac5aac6d9b8480b9ec17074a26879ad) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:33.222075 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39d757-f865-4317-b72b-9e4ca7f4af4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:33.223563 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39d757-f865-4317-b72b-9e4ca7f4af4d None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:33.230096 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 101/207] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:55:33 2022] GET /identity/v3/role_assignments?user.id=6ac5aac6d9b8480b9ec17074a26879ad&scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4&role.id=b1bd3fa77e3d4c83aa57374cb35fd21f => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:33.244454 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4925a-b7ca-4753-877d-04927bf70198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:33.245444 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4925a-b7ca-4753-877d-04927bf70198 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:33.245969 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4925a-b7ca-4753-877d-04927bf70198 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:33.246444 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4925a-b7ca-4753-877d-04927bf70198 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:33.247517 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4925a-b7ca-4753-877d-04927bf70198 None admin] RBAC: Adding query filter params (role.id=9c6920b8e6f744a6a80371b4218605fd, scope.project.id=78733ddb1b714faf83957113a15c51bc, user.id=ea62c6c957364b18945d11e0fd283e69) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:33.248090 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4925a-b7ca-4753-877d-04927bf70198 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:33.249839 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4925a-b7ca-4753-877d-04927bf70198 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:33.256807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 107/208] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:55:33 2022] GET /identity/v3/role_assignments?user.id=ea62c6c957364b18945d11e0fd283e69&scope.project.id=78733ddb1b714faf83957113a15c51bc&role.id=9c6920b8e6f744a6a80371b4218605fd => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:34.123154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02749120-17a6-4348-af69-3558740224a5 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:34.123534 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02749120-17a6-4348-af69-3558740224a5 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:34.123711 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02749120-17a6-4348-af69-3558740224a5 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:34.125196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 102/209] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:34.130665 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567951a-677a-41bd-9de2-78830566b096 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:34.131563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567951a-677a-41bd-9de2-78830566b096 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:34.132066 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567951a-677a-41bd-9de2-78830566b096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:34.151740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-7567951a-677a-41bd-9de2-78830566b096 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:34.153197 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-7567951a-677a-41bd-9de2-78830566b096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:34.163997 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:34.231754 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 108/210] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:55:34.247866 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a429c00-f46a-4dee-b28c-63cf41bf8fa6 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:34.248755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a429c00-f46a-4dee-b28c-63cf41bf8fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:34.249256 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a429c00-f46a-4dee-b28c-63cf41bf8fa6 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:34.251062 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 103/211] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:34.256379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f009f1-6483-47aa-acab-ad8d0c9e6d8a None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:34.256914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f009f1-6483-47aa-acab-ad8d0c9e6d8a None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:34.257400 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f009f1-6483-47aa-acab-ad8d0c9e6d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:34.276087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-65f009f1-6483-47aa-acab-ad8d0c9e6d8a None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:34.277497 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-65f009f1-6483-47aa-acab-ad8d0c9e6d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:34.345630 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 109/212] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:34.502663 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a429c00-f46a-4dee-b28c-63cf41bf8fa6 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:34.516182 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:34.612826 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a429c00-f46a-4dee-b28c-63cf41bf8fa6 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:34.622308 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f009f1-6483-47aa-acab-ad8d0c9e6d8a None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:34.647200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc757b-4e2b-431f-a82d-be0050290ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:34.648485 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc757b-4e2b-431f-a82d-be0050290ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:34.648775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc757b-4e2b-431f-a82d-be0050290ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:34.649046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc757b-4e2b-431f-a82d-be0050290ae1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:34.650155 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc757b-4e2b-431f-a82d-be0050290ae1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:34.651853 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dc757b-4e2b-431f-a82d-be0050290ae1 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:34.653606 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 104/213] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:55:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:34.659034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc757b-4e2b-431f-a82d-be0050290ae1 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:34.713076 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:34.739923 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f009f1-6483-47aa-acab-ad8d0c9e6d8a None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:34.752956 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:34.778203 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ff227-4d82-4ea7-ab79-72f8b3e6ff1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:34.778972 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ff227-4d82-4ea7-ab79-72f8b3e6ff1b None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:34.779376 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ff227-4d82-4ea7-ab79-72f8b3e6ff1b None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:34.779734 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1ff227-4d82-4ea7-ab79-72f8b3e6ff1b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:34.782725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc757b-4e2b-431f-a82d-be0050290ae1 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:34.790641 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1ff227-4d82-4ea7-ab79-72f8b3e6ff1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:34.795674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-cb1ff227-4d82-4ea7-ab79-72f8b3e6ff1b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:34.795674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:34.795674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:34.795674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:34.795674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:34.795674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:34.795674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:34.795674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:34.795674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:34.795674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:34.795674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:34.795674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:34.795674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:34.795674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:34.795674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:34.795674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:34.796741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:34.796741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:34.796741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:34.796741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:34.796741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:34.796741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:34.796741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:34.796741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:34.796741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:34.796741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:34.796741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:34.796741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:34.796741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:34.796741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:34.796741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:34.796741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:34.797711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:34.797711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:34.797711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:34.797711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:34.797711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:34.797711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:34.797711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:34.797711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:34.797711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:34.797711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:34.797711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:34.797711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:34.803198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 110/214] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:34.806180 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ff227-4d82-4ea7-ab79-72f8b3e6ff1b None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:34.821248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfbfd2-44c3-416c-b89b-ea6a74d558d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:34.822111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfbfd2-44c3-416c-b89b-ea6a74d558d1 None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:34.822481 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfbfd2-44c3-416c-b89b-ea6a74d558d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:34.822848 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bfbfd2-44c3-416c-b89b-ea6a74d558d1 None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:34.823650 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bfbfd2-44c3-416c-b89b-ea6a74d558d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:34.825659 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bfbfd2-44c3-416c-b89b-ea6a74d558d1 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:34.851191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 105/215] 10.222.0.36 () {60 vars in 1132 bytes} [Fri Sep 2 08:55:34 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 08:55:34.921099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1ff227-4d82-4ea7-ab79-72f8b3e6ff1b None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:34.959705 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf54ccc-6ce7-4ccb-9dfd-ae9b00acac79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:34.960374 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf54ccc-6ce7-4ccb-9dfd-ae9b00acac79 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:34.960653 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf54ccc-6ce7-4ccb-9dfd-ae9b00acac79 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:34.960920 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf54ccc-6ce7-4ccb-9dfd-ae9b00acac79 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:34.961599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf54ccc-6ce7-4ccb-9dfd-ae9b00acac79 None admin] RBAC: Adding query filter params (name=member) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:34.961896 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf54ccc-6ce7-4ccb-9dfd-ae9b00acac79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:34.963332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf54ccc-6ce7-4ccb-9dfd-ae9b00acac79 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:34.964948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:34.979279 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 111/216] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:34.982678 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfbfd2-44c3-416c-b89b-ea6a74d558d1 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:35.045967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf78f6f-3599-48d7-bc18-b0dfa8928433 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:35.046257 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf78f6f-3599-48d7-bc18-b0dfa8928433 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:35.046412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf78f6f-3599-48d7-bc18-b0dfa8928433 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:35.048135 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 112/217] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:55:35.056086 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d564a9d-5c40-4de9-a77a-a54b2071a1c5 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:35.056371 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d564a9d-5c40-4de9-a77a-a54b2071a1c5 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:35.056646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d564a9d-5c40-4de9-a77a-a54b2071a1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:35.079974 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-2d564a9d-5c40-4de9-a77a-a54b2071a1c5 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:35.081067 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-2d564a9d-5c40-4de9-a77a-a54b2071a1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:35.104911 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bfbfd2-44c3-416c-b89b-ea6a74d558d1 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:35.148845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbdc6cb-1cb8-4752-afbe-df1fe15b8c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:35.149952 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdc6cb-1cb8-4752-afbe-df1fe15b8c94 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:35.150896 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdc6cb-1cb8-4752-afbe-df1fe15b8c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:35.151425 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbdc6cb-1cb8-4752-afbe-df1fe15b8c94 None admin] PATH_INFO: `/v3/users/680233e444574c71bfdfeff4e94f799d` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:35.154436 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:35.171221 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 113/218] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:55:35.176114 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbdc6cb-1cb8-4752-afbe-df1fe15b8c94 None admin] RBAC: Authorizing `identity:get_user(user_id=680233e444574c71bfdfeff4e94f799d)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:35.177819 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbdc6cb-1cb8-4752-afbe-df1fe15b8c94 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:35.181009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 106/219] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:34 2022] GET /identity/v3/users/680233e444574c71bfdfeff4e94f799d => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:35.187197 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d564a9d-5c40-4de9-a77a-a54b2071a1c5 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:35.299767 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d564a9d-5c40-4de9-a77a-a54b2071a1c5 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:35.328441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:35.338557 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e83d35-e026-4b6d-9ba7-993678ad7979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:35.339220 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e83d35-e026-4b6d-9ba7-993678ad7979 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:35.339500 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e83d35-e026-4b6d-9ba7-993678ad7979 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:35.339768 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e83d35-e026-4b6d-9ba7-993678ad7979 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:35.402754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e92c28-2635-46bd-9ffd-eceb519c485f None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:35.403315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e92c28-2635-46bd-9ffd-eceb519c485f None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:35.403315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e92c28-2635-46bd-9ffd-eceb519c485f None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:35.404651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 107/220] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:35.409595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19607f62-df15-4cb8-84a5-326698670406 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:35.409874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19607f62-df15-4cb8-84a5-326698670406 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:35.410144 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19607f62-df15-4cb8-84a5-326698670406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:35.421090 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e83d35-e026-4b6d-9ba7-993678ad7979 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:35.422556 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e83d35-e026-4b6d-9ba7-993678ad7979 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:35.434609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-19607f62-df15-4cb8-84a5-326698670406 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:35.436805 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-19607f62-df15-4cb8-84a5-326698670406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:35.542100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 108/221] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:35.542957 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19607f62-df15-4cb8-84a5-326698670406 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:35.590766 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 114/222] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:35.596642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e83d35-e026-4b6d-9ba7-993678ad7979 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:35.646492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:35.730769 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e83d35-e026-4b6d-9ba7-993678ad7979 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:35.766366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19607f62-df15-4cb8-84a5-326698670406 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:35.767337 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511bf09-22cf-4619-aa90-9357b6cf2062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:35.768310 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511bf09-22cf-4619-aa90-9357b6cf2062 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:35.768783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511bf09-22cf-4619-aa90-9357b6cf2062 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:35.769248 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511bf09-22cf-4619-aa90-9357b6cf2062 None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:35.770585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511bf09-22cf-4619-aa90-9357b6cf2062 None admin] RBAC: Authorizing `identity:get_project(project_id=78733ddb1b714faf83957113a15c51bc)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:35.772316 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a511bf09-22cf-4619-aa90-9357b6cf2062 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:35.774720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 115/223] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:35 2022] GET /identity/v3/projects/78733ddb1b714faf83957113a15c51bc => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:35.783116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511bf09-22cf-4619-aa90-9357b6cf2062 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:35.807415 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:35.829414 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013508a-748a-41fc-878e-d846d51918b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:35.830057 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013508a-748a-41fc-878e-d846d51918b3 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:35.830337 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013508a-748a-41fc-878e-d846d51918b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:35.830608 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013508a-748a-41fc-878e-d846d51918b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:35.852164 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8013508a-748a-41fc-878e-d846d51918b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:35.857157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-8013508a-748a-41fc-878e-d846d51918b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:55:35.857157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:35.857157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:35.857157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:35.857157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:35.857157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:35.857157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:35.857157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:35.857157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:35.857157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:35.857157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:35.857157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:35.857157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:35.857157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:35.857157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:35.857157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:35.858245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:35.858245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:35.858245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:35.858245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:35.858245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:35.858245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:35.858245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:35.858245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:35.858245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:35.858245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:35.858245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:35.858245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:35.858245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:35.858245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:35.858245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:35.858245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:35.859092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:35.859092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:35.859092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:35.859092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:35.859092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:35.859092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:35.859092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:35.859092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:35.859092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:35.859092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:35.859092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:55:35.859092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:35.859092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 109/224] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:55:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:35.859092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013508a-748a-41fc-878e-d846d51918b3 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:35.916387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:35.925857 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511bf09-22cf-4619-aa90-9357b6cf2062 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:35.963482 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8013508a-748a-41fc-878e-d846d51918b3 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:35.970258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc03b86-3598-4f55-9ab4-f770d66ba6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:35.971384 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc03b86-3598-4f55-9ab4-f770d66ba6d4 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:35.971667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc03b86-3598-4f55-9ab4-f770d66ba6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:35.971933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc03b86-3598-4f55-9ab4-f770d66ba6d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:35.972657 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc03b86-3598-4f55-9ab4-f770d66ba6d4 None admin] RBAC: Adding query filter params (role.id=08c2a54a85ef4bb9892a5769ef8a5d95, scope.project.id=78733ddb1b714faf83957113a15c51bc, user.id=680233e444574c71bfdfeff4e94f799d) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:35.972957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc03b86-3598-4f55-9ab4-f770d66ba6d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:35.974387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc03b86-3598-4f55-9ab4-f770d66ba6d4 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:35.980247 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 116/225] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:55:35 2022] GET /identity/v3/role_assignments?user.id=680233e444574c71bfdfeff4e94f799d&scope.project.id=78733ddb1b714faf83957113a15c51bc&role.id=08c2a54a85ef4bb9892a5769ef8a5d95 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:35.981112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc03b86-3598-4f55-9ab4-f770d66ba6d4 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:36.009696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04c21b-e614-4442-9559-381fa3e5e3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:36.010650 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04c21b-e614-4442-9559-381fa3e5e3a9 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:36.010933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04c21b-e614-4442-9559-381fa3e5e3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:36.011240 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04c21b-e614-4442-9559-381fa3e5e3a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:36.027227 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd04c21b-e614-4442-9559-381fa3e5e3a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:36.034341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-bd04c21b-e614-4442-9559-381fa3e5e3a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:55:36.034341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:36.034341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:36.034341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:36.034341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:36.034341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:36.034341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:36.034341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:36.034341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:36.034341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:36.034341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:36.034341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:36.034341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:36.034341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:36.034341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:36.034341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:36.035592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:36.035592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:36.035592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:36.035592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:36.035592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:36.035592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:36.035592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:36.035592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:36.035592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:36.035592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:36.035592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:36.035592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:36.035592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:36.035592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:36.035592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:36.035592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:36.036395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:36.036395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:36.036395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:36.036395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:36.036395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:36.036395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:36.036395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:36.036395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:36.036395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:36.036395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:36.036395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:55:36.036395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:36.036395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 110/226] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:55:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:36.043807 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04c21b-e614-4442-9559-381fa3e5e3a9 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:36.108039 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:36.108922 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc03b86-3598-4f55-9ab4-f770d66ba6d4 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:36.146389 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff3bb8-cd06-4134-8172-d391d96caedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:36.146993 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff3bb8-cd06-4134-8172-d391d96caedd None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:36.147448 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff3bb8-cd06-4134-8172-d391d96caedd None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:36.148304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff3bb8-cd06-4134-8172-d391d96caedd None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:36.149191 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff3bb8-cd06-4134-8172-d391d96caedd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:36.149462 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff3bb8-cd06-4134-8172-d391d96caedd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:36.150980 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff3bb8-cd06-4134-8172-d391d96caedd None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:36.160801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 117/227] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:36.169273 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff3bb8-cd06-4134-8172-d391d96caedd None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:36.185096 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04c21b-e614-4442-9559-381fa3e5e3a9 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:36.217007 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:36.243204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41fae4-ff12-452b-9d19-9b041067bcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:36.244180 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41fae4-ff12-452b-9d19-9b041067bcc4 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:36.244654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41fae4-ff12-452b-9d19-9b041067bcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:36.245116 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41fae4-ff12-452b-9d19-9b041067bcc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:36.245998 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f41fae4-ff12-452b-9d19-9b041067bcc4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:36.246518 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f41fae4-ff12-452b-9d19-9b041067bcc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:36.248117 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f41fae4-ff12-452b-9d19-9b041067bcc4 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:36.258990 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 111/228] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:55:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:36.265666 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41fae4-ff12-452b-9d19-9b041067bcc4 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:36.305668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff3bb8-cd06-4134-8172-d391d96caedd None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:36.343255 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f8d03-33d8-46f5-b7d5-c2c68443050d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:36.343255 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f8d03-33d8-46f5-b7d5-c2c68443050d None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:36.343255 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f8d03-33d8-46f5-b7d5-c2c68443050d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:36.343255 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f8d03-33d8-46f5-b7d5-c2c68443050d None admin] PATH_INFO: `/v3/users/ea62c6c957364b18945d11e0fd283e69` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:36.347979 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f8d03-33d8-46f5-b7d5-c2c68443050d None admin] RBAC: Authorizing `identity:get_user(user_id=ea62c6c957364b18945d11e0fd283e69)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:36.349447 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f8d03-33d8-46f5-b7d5-c2c68443050d None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:36.351292 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 118/229] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:36 2022] GET /identity/v3/users/ea62c6c957364b18945d11e0fd283e69 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:36.363163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f8d03-33d8-46f5-b7d5-c2c68443050d None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:36.428420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41fae4-ff12-452b-9d19-9b041067bcc4 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:36.432825 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:36.471640 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f3db2-b699-4e06-8fdc-af332b1424b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:36.472280 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f3db2-b699-4e06-8fdc-af332b1424b2 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:36.472554 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f3db2-b699-4e06-8fdc-af332b1424b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:36.472846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f3db2-b699-4e06-8fdc-af332b1424b2 None admin] PATH_INFO: `/v3/users/ea62c6c957364b18945d11e0fd283e69` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:36.473963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f3db2-b699-4e06-8fdc-af332b1424b2 None admin] RBAC: Authorizing `identity:get_user(user_id=ea62c6c957364b18945d11e0fd283e69)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:36.475447 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f3db2-b699-4e06-8fdc-af332b1424b2 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:36.477108 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 112/230] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:36 2022] GET /identity/v3/users/ea62c6c957364b18945d11e0fd283e69 => generated 283 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:36.482490 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f3db2-b699-4e06-8fdc-af332b1424b2 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:36.579454 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:36.613270 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f8d03-33d8-46f5-b7d5-c2c68443050d None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:36.632832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f3db2-b699-4e06-8fdc-af332b1424b2 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:36.672464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9579e8c-f545-46f8-bd29-4a92a11895f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:36.673244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9579e8c-f545-46f8-bd29-4a92a11895f0 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:36.673524 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9579e8c-f545-46f8-bd29-4a92a11895f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:36.673795 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9579e8c-f545-46f8-bd29-4a92a11895f0 None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:36.674914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9579e8c-f545-46f8-bd29-4a92a11895f0 None admin] RBAC: Authorizing `identity:get_project(project_id=78733ddb1b714faf83957113a15c51bc)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:36.676755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9579e8c-f545-46f8-bd29-4a92a11895f0 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:36.679730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 113/231] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:36 2022] GET /identity/v3/projects/78733ddb1b714faf83957113a15c51bc => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:36.683758 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af142e7d-bbbc-4e81-837e-298b2b17cc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:36.684480 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af142e7d-bbbc-4e81-837e-298b2b17cc3c None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:36.685405 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9579e8c-f545-46f8-bd29-4a92a11895f0 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:36.688311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af142e7d-bbbc-4e81-837e-298b2b17cc3c None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:36.688593 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af142e7d-bbbc-4e81-837e-298b2b17cc3c None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:36.690546 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af142e7d-bbbc-4e81-837e-298b2b17cc3c None admin] RBAC: Authorizing `identity:get_project(project_id=78733ddb1b714faf83957113a15c51bc)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:36.692027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af142e7d-bbbc-4e81-837e-298b2b17cc3c None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:36.693646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 119/232] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:36 2022] GET /identity/v3/projects/78733ddb1b714faf83957113a15c51bc => generated 302 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:36.700376 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af142e7d-bbbc-4e81-837e-298b2b17cc3c None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:36.756381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:36.841174 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9579e8c-f545-46f8-bd29-4a92a11895f0 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:36.876937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a06dd-d188-4013-a416-363287c72c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:36.877525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a06dd-d188-4013-a416-363287c72c0e None admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:36.877802 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a06dd-d188-4013-a416-363287c72c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:36.878074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a06dd-d188-4013-a416-363287c72c0e None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc/users/ea62c6c957364b18945d11e0fd283e69/roles/9c6920b8e6f744a6a80371b4218605fd` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:36.880367 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a06dd-d188-4013-a416-363287c72c0e None admin] RBAC: Authorizing `identity:create_grant(project_id=78733ddb1b714faf83957113a15c51bc, user_id=ea62c6c957364b18945d11e0fd283e69, role_id=9c6920b8e6f744a6a80371b4218605fd)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:36.881839 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a06dd-d188-4013-a416-363287c72c0e None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:36.886175 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af142e7d-bbbc-4e81-837e-298b2b17cc3c None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:36.913522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 114/233] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:55:36 2022] PUT /identity/v3/projects/78733ddb1b714faf83957113a15c51bc/users/ea62c6c957364b18945d11e0fd283e69/roles/9c6920b8e6f744a6a80371b4218605fd => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 08:55:36.933202 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d22e7-ab84-43f8-a6a1-112289ba54d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:36.933886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d22e7-ab84-43f8-a6a1-112289ba54d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:36.934180 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d22e7-ab84-43f8-a6a1-112289ba54d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:36.934432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d22e7-ab84-43f8-a6a1-112289ba54d3 None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc/users/ea62c6c957364b18945d11e0fd283e69/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:36.940670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d22e7-ab84-43f8-a6a1-112289ba54d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=78733ddb1b714faf83957113a15c51bc, user_id=ea62c6c957364b18945d11e0fd283e69, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:36.942149 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d22e7-ab84-43f8-a6a1-112289ba54d3 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:36.978147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 120/234] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:55:36 2022] PUT /identity/v3/projects/78733ddb1b714faf83957113a15c51bc/users/ea62c6c957364b18945d11e0fd283e69/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:55:37.324421 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c53306-5f6c-4b42-bfbe-42947383e1d8 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:37.325013 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c53306-5f6c-4b42-bfbe-42947383e1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:37.325480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c53306-5f6c-4b42-bfbe-42947383e1d8 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:37.327335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 115/235] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:37.332435 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9334834-50a7-41cb-b16f-21b2172c8f81 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:37.333302 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9334834-50a7-41cb-b16f-21b2172c8f81 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:37.333767 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9334834-50a7-41cb-b16f-21b2172c8f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:37.358665 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-e9334834-50a7-41cb-b16f-21b2172c8f81 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:37.359880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-e9334834-50a7-41cb-b16f-21b2172c8f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:37.375359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:37.401432 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c6fa5-0d22-4621-af6c-ea73dd0c6d13 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:37.401730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c6fa5-0d22-4621-af6c-ea73dd0c6d13 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:37.401982 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c6fa5-0d22-4621-af6c-ea73dd0c6d13 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:37.404196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 116/236] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:37.409962 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a33666-fec4-4cdf-bfcb-adbf60f86fde None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:37.410241 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a33666-fec4-4cdf-bfcb-adbf60f86fde None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:37.410514 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a33666-fec4-4cdf-bfcb-adbf60f86fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:37.446437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-93a33666-fec4-4cdf-bfcb-adbf60f86fde None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:37.448689 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-93a33666-fec4-4cdf-bfcb-adbf60f86fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:37.488835 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 121/237] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:37.536950 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 117/238] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:55:37.819792 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9334834-50a7-41cb-b16f-21b2172c8f81 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:37.897958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a33666-fec4-4cdf-bfcb-adbf60f86fde None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:37.953626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:37.976522 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9334834-50a7-41cb-b16f-21b2172c8f81 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:38.045182 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f1364-7d85-428b-87c2-21b420f0d1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:38.046180 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f1364-7d85-428b-87c2-21b420f0d1fd None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:38.046846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f1364-7d85-428b-87c2-21b420f0d1fd None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:38.048185 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f1364-7d85-428b-87c2-21b420f0d1fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:38.050386 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8f1364-7d85-428b-87c2-21b420f0d1fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:38.053063 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8f1364-7d85-428b-87c2-21b420f0d1fd None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:38.055575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 122/239] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:55:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:38.062969 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f1364-7d85-428b-87c2-21b420f0d1fd None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:38.083173 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a33666-fec4-4cdf-bfcb-adbf60f86fde None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:38.096656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:38.110908 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9370fb-7861-4bc8-b20e-3292bbe55376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:38.111573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9370fb-7861-4bc8-b20e-3292bbe55376 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:38.111854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9370fb-7861-4bc8-b20e-3292bbe55376 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:38.119094 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9370fb-7861-4bc8-b20e-3292bbe55376 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:38.123562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9370fb-7861-4bc8-b20e-3292bbe55376 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:38.128732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-1a9370fb-7861-4bc8-b20e-3292bbe55376 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 08:55:38.128732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:38.128732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:38.128732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:38.128732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:38.128732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:38.128732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:38.128732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:38.128732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:38.128732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:38.128732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:38.128732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:38.128732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:38.128732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:38.128732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:38.128732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:38.129607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:38.129607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:38.129607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:38.129607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:38.129607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:38.129607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:38.129607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:38.129607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:38.129607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:38.129607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:38.129607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:38.129607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:38.129607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:38.129607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:38.129607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:38.129607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:38.130411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:38.130411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:38.130411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:38.130411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:38.130411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:38.130411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:38.130411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:38.130411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:38.130411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:38.130411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:38.130411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 08:55:38.130411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:38.142080 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 118/240] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:38.142080 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9370fb-7861-4bc8-b20e-3292bbe55376 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:38.274388 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8f1364-7d85-428b-87c2-21b420f0d1fd None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:38.305262 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:38.323985 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9370fb-7861-4bc8-b20e-3292bbe55376 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:38.338859 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564ef38-ab80-4407-8b53-0911891d9929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:38.339464 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564ef38-ab80-4407-8b53-0911891d9929 None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:38.339691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564ef38-ab80-4407-8b53-0911891d9929 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:38.339961 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564ef38-ab80-4407-8b53-0911891d9929 None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:38.343004 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564ef38-ab80-4407-8b53-0911891d9929 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:38.343004 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564ef38-ab80-4407-8b53-0911891d9929 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:38.382320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 123/241] 10.222.0.36 () {60 vars in 1132 bytes} [Fri Sep 2 08:55:38 2022] POST /identity/v3/users => generated 293 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:55:38.388333 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ad36a-d1de-4f91-a41d-6eeccbefc15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:38.389018 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ad36a-d1de-4f91-a41d-6eeccbefc15c None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:38.389300 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ad36a-d1de-4f91-a41d-6eeccbefc15c None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:38.389587 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ad36a-d1de-4f91-a41d-6eeccbefc15c None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:38.390317 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ad36a-d1de-4f91-a41d-6eeccbefc15c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:38.390625 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ad36a-d1de-4f91-a41d-6eeccbefc15c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:38.392077 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048ad36a-d1de-4f91-a41d-6eeccbefc15c None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:38.402089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 119/242] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:38.408089 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564ef38-ab80-4407-8b53-0911891d9929 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:38.433202 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf623e-3fea-4663-a08a-7abe78af9ac9 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:38.433530 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf623e-3fea-4663-a08a-7abe78af9ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:38.433807 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf623e-3fea-4663-a08a-7abe78af9ac9 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:38.435610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 120/243] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:55:38.441715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ec5c2-6bd5-447d-bb3a-4f83aaacdba0 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:38.442034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ec5c2-6bd5-447d-bb3a-4f83aaacdba0 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:38.442318 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ec5c2-6bd5-447d-bb3a-4f83aaacdba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:38.475830 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-884ec5c2-6bd5-447d-bb3a-4f83aaacdba0 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:38.481055 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-884ec5c2-6bd5-447d-bb3a-4f83aaacdba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:38.551442 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:38.597848 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564ef38-ab80-4407-8b53-0911891d9929 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:38.604265 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 121/244] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:55:38.642684 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c1452-c5b6-4cbf-bb80-3334920ade31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:38.643352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c1452-c5b6-4cbf-bb80-3334920ade31 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:38.643630 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c1452-c5b6-4cbf-bb80-3334920ade31 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:38.644036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495c1452-c5b6-4cbf-bb80-3334920ade31 None admin] PATH_INFO: `/v3/users/39c417d520664dbf9f4b95841e4903b0` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:38.662002 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495c1452-c5b6-4cbf-bb80-3334920ade31 None admin] RBAC: Authorizing `identity:get_user(user_id=39c417d520664dbf9f4b95841e4903b0)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:38.664129 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495c1452-c5b6-4cbf-bb80-3334920ade31 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:38.665952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 124/245] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:38 2022] GET /identity/v3/users/39c417d520664dbf9f4b95841e4903b0 => generated 297 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:38.671542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ec5c2-6bd5-447d-bb3a-4f83aaacdba0 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:38.759953 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:38.775919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ec5c2-6bd5-447d-bb3a-4f83aaacdba0 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:38.811189 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6536d1-6d5e-49b2-a681-ef211012948c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:38.811813 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6536d1-6d5e-49b2-a681-ef211012948c None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:38.812089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6536d1-6d5e-49b2-a681-ef211012948c None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:38.812358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6536d1-6d5e-49b2-a681-ef211012948c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:38.880230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6536d1-6d5e-49b2-a681-ef211012948c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:38.881731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6536d1-6d5e-49b2-a681-ef211012948c None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:38.922276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c1452-c5b6-4cbf-bb80-3334920ade31 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:38.973901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:38.996506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 122/246] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:39.008140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6536d1-6d5e-49b2-a681-ef211012948c None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:39.058727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495c1452-c5b6-4cbf-bb80-3334920ade31 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:39.093479 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:39.104912 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ed491-72d0-4293-872d-ec5c04b77611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:39.105698 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ed491-72d0-4293-872d-ec5c04b77611 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:39.106335 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ed491-72d0-4293-872d-ec5c04b77611 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:39.106710 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ed491-72d0-4293-872d-ec5c04b77611 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:39.120829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ed491-72d0-4293-872d-ec5c04b77611 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:39.127951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6536d1-6d5e-49b2-a681-ef211012948c None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:39.128328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-669ed491-72d0-4293-872d-ec5c04b77611 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:39.128328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:39.128328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:39.128328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:39.128328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:39.128328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:39.128328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:39.128328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:39.128328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:39.128328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:39.128328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:39.128328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:39.128328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:39.128328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:39.128328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:39.128328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:39.129198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:39.129198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:39.129198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:39.129198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:39.129198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:39.129198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:39.129198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:39.129198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:39.129198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:39.129198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:39.129198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:39.129198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:39.129198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:39.129198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:39.129198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:39.129198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:39.129975 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:39.129975 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:39.129975 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:39.129975 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:39.129975 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:39.129975 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:39.129975 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:39.129975 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:39.129975 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:39.129975 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:39.129975 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:39.129975 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:39.129975 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 125/247] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:55:39.137682 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ed491-72d0-4293-872d-ec5c04b77611 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:39.177138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2eafc8-d32b-46af-b809-ea122c81ce69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:39.177787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2eafc8-d32b-46af-b809-ea122c81ce69 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:39.178075 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2eafc8-d32b-46af-b809-ea122c81ce69 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:39.178380 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2eafc8-d32b-46af-b809-ea122c81ce69 None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:39.179564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2eafc8-d32b-46af-b809-ea122c81ce69 None admin] RBAC: Authorizing `identity:get_project(project_id=78733ddb1b714faf83957113a15c51bc)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:39.181019 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2eafc8-d32b-46af-b809-ea122c81ce69 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:39.186217 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 123/248] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:39 2022] GET /identity/v3/projects/78733ddb1b714faf83957113a15c51bc => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:39.186217 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b2616-40fb-42f3-9eb9-5b0de3b025f9 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:39.186217 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b2616-40fb-42f3-9eb9-5b0de3b025f9 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:39.186217 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b2616-40fb-42f3-9eb9-5b0de3b025f9 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:39.186557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 124/249] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:39.193098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e6cb8-ff16-4d54-8b82-c58ae0638d70 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:39.193379 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e6cb8-ff16-4d54-8b82-c58ae0638d70 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:39.193652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e6cb8-ff16-4d54-8b82-c58ae0638d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:39.216977 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-041e6cb8-ff16-4d54-8b82-c58ae0638d70 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:39.218095 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-041e6cb8-ff16-4d54-8b82-c58ae0638d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:39.261236 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ed491-72d0-4293-872d-ec5c04b77611 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:39.271457 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:39.295580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 125/250] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:55:39.296682 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e6cb8-ff16-4d54-8b82-c58ae0638d70 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:39.299449 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210f202-05a2-4c3f-ad4d-9ecfa30ccdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:39.300172 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210f202-05a2-4c3f-ad4d-9ecfa30ccdef None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:39.300553 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210f202-05a2-4c3f-ad4d-9ecfa30ccdef None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:39.300930 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210f202-05a2-4c3f-ad4d-9ecfa30ccdef None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:39.301698 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210f202-05a2-4c3f-ad4d-9ecfa30ccdef None admin] RBAC: Adding query filter params (name=member) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:39.302102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210f202-05a2-4c3f-ad4d-9ecfa30ccdef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:39.304125 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210f202-05a2-4c3f-ad4d-9ecfa30ccdef None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:39.313553 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 126/251] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:39.323371 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210f202-05a2-4c3f-ad4d-9ecfa30ccdef None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:39.387985 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:39.445352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210f202-05a2-4c3f-ad4d-9ecfa30ccdef None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:39.485666 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581e4ee-c988-448d-97ad-93888bd97573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:39.486302 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581e4ee-c988-448d-97ad-93888bd97573 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:39.486586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581e4ee-c988-448d-97ad-93888bd97573 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:39.486860 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581e4ee-c988-448d-97ad-93888bd97573 None admin] PATH_INFO: `/v3/users/680233e444574c71bfdfeff4e94f799d` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:39.488033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e581e4ee-c988-448d-97ad-93888bd97573 None admin] RBAC: Authorizing `identity:get_user(user_id=680233e444574c71bfdfeff4e94f799d)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:39.490107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e581e4ee-c988-448d-97ad-93888bd97573 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:39.492735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 127/252] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:39 2022] GET /identity/v3/users/680233e444574c71bfdfeff4e94f799d => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:39.493453 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e6cb8-ff16-4d54-8b82-c58ae0638d70 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:39.498045 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581e4ee-c988-448d-97ad-93888bd97573 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:39.556671 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19a618-8228-4f43-8fac-1167c1f77db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:39.557288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19a618-8228-4f43-8fac-1167c1f77db3 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:39.557565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19a618-8228-4f43-8fac-1167c1f77db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:39.557836 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19a618-8228-4f43-8fac-1167c1f77db3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:39.558679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19a618-8228-4f43-8fac-1167c1f77db3 None admin] RBAC: Adding query filter params (role.id=b1bd3fa77e3d4c83aa57374cb35fd21f, scope.project.id=78733ddb1b714faf83957113a15c51bc, user.id=39c417d520664dbf9f4b95841e4903b0) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:39.558993 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19a618-8228-4f43-8fac-1167c1f77db3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:39.560561 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19a618-8228-4f43-8fac-1167c1f77db3 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:39.574597 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 126/253] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:55:39 2022] GET /identity/v3/role_assignments?user.id=39c417d520664dbf9f4b95841e4903b0&scope.project.id=78733ddb1b714faf83957113a15c51bc&role.id=b1bd3fa77e3d4c83aa57374cb35fd21f => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:39.651951 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e581e4ee-c988-448d-97ad-93888bd97573 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:39.682666 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:39.707954 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0c59b-5642-43d7-9dba-0ca0a098c333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:39.708722 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0c59b-5642-43d7-9dba-0ca0a098c333 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:39.709129 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0c59b-5642-43d7-9dba-0ca0a098c333 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:39.709411 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0c59b-5642-43d7-9dba-0ca0a098c333 None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:39.710719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0c59b-5642-43d7-9dba-0ca0a098c333 None admin] RBAC: Authorizing `identity:get_project(project_id=78733ddb1b714faf83957113a15c51bc)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:39.712231 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0c59b-5642-43d7-9dba-0ca0a098c333 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:39.714228 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 128/254] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:39 2022] GET /identity/v3/projects/78733ddb1b714faf83957113a15c51bc => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:39.720000 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0c59b-5642-43d7-9dba-0ca0a098c333 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:39.720170 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19a618-8228-4f43-8fac-1167c1f77db3 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:39.822147 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:39.840445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0c59b-5642-43d7-9dba-0ca0a098c333 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:39.880656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d8018-8114-4aac-bb79-18f9ec308950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:39.881323 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d8018-8114-4aac-bb79-18f9ec308950 None admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:39.881638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d8018-8114-4aac-bb79-18f9ec308950 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:39.881915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d8018-8114-4aac-bb79-18f9ec308950 None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc/users/680233e444574c71bfdfeff4e94f799d/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:39.885005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d8018-8114-4aac-bb79-18f9ec308950 None admin] RBAC: Authorizing `identity:create_grant(project_id=78733ddb1b714faf83957113a15c51bc, user_id=680233e444574c71bfdfeff4e94f799d, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:39.886468 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d8018-8114-4aac-bb79-18f9ec308950 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:39.903412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 129/255] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:55:39 2022] PUT /identity/v3/projects/78733ddb1b714faf83957113a15c51bc/users/680233e444574c71bfdfeff4e94f799d/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 08:55:39.905176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2343e8-e55b-449b-8116-44fc3e76afea None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:39.905673 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2343e8-e55b-449b-8116-44fc3e76afea None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:39.906086 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2343e8-e55b-449b-8116-44fc3e76afea None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:39.907734 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 130/256] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:39.915337 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa47304-2468-423d-951d-b95e503b1d31 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:39.915738 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa47304-2468-423d-951d-b95e503b1d31 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:39.915947 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa47304-2468-423d-951d-b95e503b1d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:39.921971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19a618-8228-4f43-8fac-1167c1f77db3 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:39.940582 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-cfa47304-2468-423d-951d-b95e503b1d31 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:39.942576 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-cfa47304-2468-423d-951d-b95e503b1d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:39.996941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:40.008664 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f421c-0323-4a06-a8af-8e1a29b462bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:40.009697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f421c-0323-4a06-a8af-8e1a29b462bf None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:40.010280 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f421c-0323-4a06-a8af-8e1a29b462bf None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:40.010731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f421c-0323-4a06-a8af-8e1a29b462bf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:40.031713 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f421c-0323-4a06-a8af-8e1a29b462bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:40.042238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-cd0f421c-0323-4a06-a8af-8e1a29b462bf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:55:40.042238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:40.042238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:40.042238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:40.042238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:40.042238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:40.042238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:40.042238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:40.042238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:40.042238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:40.042238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:40.042238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:40.042238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:40.042238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:40.042238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:40.042238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:40.043266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:40.043266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:40.043266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:40.043266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:40.043266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:40.043266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:40.043266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:40.043266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:40.043266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:40.043266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:40.043266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:40.043266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:40.043266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:40.043266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:40.043266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:40.043266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:40.044089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:40.044089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:40.044089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:40.044089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:40.044089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:40.044089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:40.044089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:40.044089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:40.044089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:40.044089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:40.044089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:55:40.044089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:40.044692 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 127/257] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:55:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:40.049448 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 131/258] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:40.056311 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f421c-0323-4a06-a8af-8e1a29b462bf None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:40.236914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f421c-0323-4a06-a8af-8e1a29b462bf None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:40.244616 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:40.281928 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8cfaa-4d2c-4ffc-bae6-583d10476e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:40.282551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8cfaa-4d2c-4ffc-bae6-583d10476e5e None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:40.282883 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8cfaa-4d2c-4ffc-bae6-583d10476e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:40.283183 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8cfaa-4d2c-4ffc-bae6-583d10476e5e None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:40.283860 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8cfaa-4d2c-4ffc-bae6-583d10476e5e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:40.284162 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8cfaa-4d2c-4ffc-bae6-583d10476e5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:40.285529 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8cfaa-4d2c-4ffc-bae6-583d10476e5e None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:40.294819 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 128/259] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:55:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:40.300097 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa47304-2468-423d-951d-b95e503b1d31 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:40.436374 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa47304-2468-423d-951d-b95e503b1d31 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:40.437622 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8cfaa-4d2c-4ffc-bae6-583d10476e5e None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:40.507692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10506d-8535-4322-9a0e-027a60454802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:40.509186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10506d-8535-4322-9a0e-027a60454802 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:40.509936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10506d-8535-4322-9a0e-027a60454802 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:40.510581 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10506d-8535-4322-9a0e-027a60454802 None admin] PATH_INFO: `/v3/users/ea62c6c957364b18945d11e0fd283e69` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:40.514017 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10506d-8535-4322-9a0e-027a60454802 None admin] RBAC: Authorizing `identity:get_user(user_id=ea62c6c957364b18945d11e0fd283e69)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:40.515735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d10506d-8535-4322-9a0e-027a60454802 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:40.520355 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 132/260] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:40 2022] GET /identity/v3/users/ea62c6c957364b18945d11e0fd283e69 => generated 283 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:40.527739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10506d-8535-4322-9a0e-027a60454802 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:40.584442 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:40.636693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10506d-8535-4322-9a0e-027a60454802 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:40.669014 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8cfaa-4d2c-4ffc-bae6-583d10476e5e None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:40.674055 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a5efb-972b-486a-8f52-4b04e2703533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:40.674936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a5efb-972b-486a-8f52-4b04e2703533 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:40.675534 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a5efb-972b-486a-8f52-4b04e2703533 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:40.676014 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a5efb-972b-486a-8f52-4b04e2703533 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:40.732261 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0c07f-e2a2-49e0-a2db-5974a9f80571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:40.732911 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0c07f-e2a2-49e0-a2db-5974a9f80571 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:40.733184 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0c07f-e2a2-49e0-a2db-5974a9f80571 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:40.733465 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0c07f-e2a2-49e0-a2db-5974a9f80571 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:40.749644 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a5efb-972b-486a-8f52-4b04e2703533 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:40.750750 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f0c07f-e2a2-49e0-a2db-5974a9f80571 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:40.751698 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a5efb-972b-486a-8f52-4b04e2703533 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:40.756338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-27f0c07f-e2a2-49e0-a2db-5974a9f80571 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:55:40.756338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:40.756338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:40.756338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:40.756338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:40.756338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:40.756338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:40.756338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:40.756338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:40.756338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:40.756338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:40.756338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:40.756338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:40.756338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:40.756338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:40.756338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:40.757251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:40.757251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:40.757251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:40.757251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:40.757251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:40.757251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:40.757251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:40.757251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:40.757251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:40.757251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:40.757251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:40.757251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:40.757251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:40.757251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:40.757251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:40.757251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:40.758091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:40.758091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:40.758091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:40.758091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:40.758091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:40.758091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:40.758091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:40.758091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:40.758091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:40.758091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:40.758091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:55:40.758091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:40.758091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 129/261] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:55:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:55:40.782241 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0c07f-e2a2-49e0-a2db-5974a9f80571 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:40.807648 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:40.827487 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:40.867271 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 133/262] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:40.873224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a5efb-972b-486a-8f52-4b04e2703533 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:40.954320 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0c07f-e2a2-49e0-a2db-5974a9f80571 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:41.009637 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a5efb-972b-486a-8f52-4b04e2703533 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:41.026704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087af09-2ad5-426c-a035-7ef95e24ad51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:41.027718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087af09-2ad5-426c-a035-7ef95e24ad51 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:41.030140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087af09-2ad5-426c-a035-7ef95e24ad51 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:41.030622 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087af09-2ad5-426c-a035-7ef95e24ad51 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:41.031623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087af09-2ad5-426c-a035-7ef95e24ad51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:41.032914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087af09-2ad5-426c-a035-7ef95e24ad51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:41.035262 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087af09-2ad5-426c-a035-7ef95e24ad51 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:41.059205 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 130/263] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:41.072843 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087af09-2ad5-426c-a035-7ef95e24ad51 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:41.087923 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb248f9b-a139-45e4-a401-af34ebc83eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:41.088955 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb248f9b-a139-45e4-a401-af34ebc83eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:41.089432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb248f9b-a139-45e4-a401-af34ebc83eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:41.089925 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb248f9b-a139-45e4-a401-af34ebc83eb9 None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:41.091996 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb248f9b-a139-45e4-a401-af34ebc83eb9 None admin] RBAC: Authorizing `identity:get_project(project_id=78733ddb1b714faf83957113a15c51bc)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:41.095359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb248f9b-a139-45e4-a401-af34ebc83eb9 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:41.098354 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 134/264] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:40 2022] GET /identity/v3/projects/78733ddb1b714faf83957113a15c51bc => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:41.114850 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb248f9b-a139-45e4-a401-af34ebc83eb9 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:41.135405 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:41.234765 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:41.241010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087af09-2ad5-426c-a035-7ef95e24ad51 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:41.286945 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb248f9b-a139-45e4-a401-af34ebc83eb9 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:41.308962 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d38eff-4fcd-4a0c-a910-504bcab9244b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:41.309607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d38eff-4fcd-4a0c-a910-504bcab9244b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:41.309893 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d38eff-4fcd-4a0c-a910-504bcab9244b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:41.310172 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d38eff-4fcd-4a0c-a910-504bcab9244b None admin] PATH_INFO: `/v3/users/ea62c6c957364b18945d11e0fd283e69` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:41.312859 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d38eff-4fcd-4a0c-a910-504bcab9244b None admin] RBAC: Authorizing `identity:get_user(user_id=ea62c6c957364b18945d11e0fd283e69)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:41.314312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d38eff-4fcd-4a0c-a910-504bcab9244b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:41.321393 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 131/265] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:41 2022] GET /identity/v3/users/ea62c6c957364b18945d11e0fd283e69 => generated 283 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:41.344003 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d38eff-4fcd-4a0c-a910-504bcab9244b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:41.356169 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdfa65c-2c02-429b-8d11-b6f6303dc428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:41.356792 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfa65c-2c02-429b-8d11-b6f6303dc428 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:41.357069 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfa65c-2c02-429b-8d11-b6f6303dc428 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:41.357337 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfa65c-2c02-429b-8d11-b6f6303dc428 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:41.358126 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdfa65c-2c02-429b-8d11-b6f6303dc428 None admin] RBAC: Adding query filter params (role.id=9c6920b8e6f744a6a80371b4218605fd, scope.project.id=78733ddb1b714faf83957113a15c51bc, user.id=ea62c6c957364b18945d11e0fd283e69) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:41.358435 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdfa65c-2c02-429b-8d11-b6f6303dc428 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:41.361970 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdfa65c-2c02-429b-8d11-b6f6303dc428 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:41.382851 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 135/266] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:55:41 2022] GET /identity/v3/role_assignments?user.id=ea62c6c957364b18945d11e0fd283e69&scope.project.id=78733ddb1b714faf83957113a15c51bc&role.id=9c6920b8e6f744a6a80371b4218605fd => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:41.503237 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d38eff-4fcd-4a0c-a910-504bcab9244b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:41.540776 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9e452-17be-4e02-9529-e26007548c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:41.541323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9e452-17be-4e02-9529-e26007548c16 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:41.541605 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9e452-17be-4e02-9529-e26007548c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:41.541881 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd9e452-17be-4e02-9529-e26007548c16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:41.574485 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:41.624394 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9e452-17be-4e02-9529-e26007548c16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:41.625873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd9e452-17be-4e02-9529-e26007548c16 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:41.670455 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ddf34-3c37-4391-8301-6db29cb1bde0 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:41.670984 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ddf34-3c37-4391-8301-6db29cb1bde0 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:41.670984 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ddf34-3c37-4391-8301-6db29cb1bde0 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:41.672587 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 136/267] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:41.677671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cceaab-5f51-434c-9451-460a11919e73 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:41.677980 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cceaab-5f51-434c-9451-460a11919e73 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:41.678788 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cceaab-5f51-434c-9451-460a11919e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:41.700517 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-66cceaab-5f51-434c-9451-460a11919e73 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:41.701711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-66cceaab-5f51-434c-9451-460a11919e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:41.738856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 132/268] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:41.752097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9e452-17be-4e02-9529-e26007548c16 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:41.784485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 137/269] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:55:41.824735 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:41.875953 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd9e452-17be-4e02-9529-e26007548c16 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:41.916691 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c21a6d-f790-4577-873f-3044a264ac8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:41.917330 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c21a6d-f790-4577-873f-3044a264ac8f None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:41.917610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c21a6d-f790-4577-873f-3044a264ac8f None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:41.917882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c21a6d-f790-4577-873f-3044a264ac8f None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:41.919909 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c21a6d-f790-4577-873f-3044a264ac8f None admin] RBAC: Authorizing `identity:get_project(project_id=78733ddb1b714faf83957113a15c51bc)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:41.921370 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c21a6d-f790-4577-873f-3044a264ac8f None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:41.923384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 133/270] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:41 2022] GET /identity/v3/projects/78733ddb1b714faf83957113a15c51bc => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:41.929466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cceaab-5f51-434c-9451-460a11919e73 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:41.938509 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:42.054616 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cceaab-5f51-434c-9451-460a11919e73 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:42.095053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d60454-a318-49e6-93c2-ba4ea07fedfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:42.095612 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d60454-a318-49e6-93c2-ba4ea07fedfa None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:42.095890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d60454-a318-49e6-93c2-ba4ea07fedfa None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:42.096158 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d60454-a318-49e6-93c2-ba4ea07fedfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:42.096879 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d60454-a318-49e6-93c2-ba4ea07fedfa None admin] RBAC: Adding query filter params (role.id=a0a76de72fba4036bc1e3e64df717c7f, scope.project.id=78733ddb1b714faf83957113a15c51bc, user.id=ea62c6c957364b18945d11e0fd283e69) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:42.097181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d60454-a318-49e6-93c2-ba4ea07fedfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:42.098631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d60454-a318-49e6-93c2-ba4ea07fedfa None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:42.104855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 138/271] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:55:41 2022] GET /identity/v3/role_assignments?user.id=ea62c6c957364b18945d11e0fd283e69&scope.project.id=78733ddb1b714faf83957113a15c51bc&role.id=a0a76de72fba4036bc1e3e64df717c7f => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:42.122463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c21a6d-f790-4577-873f-3044a264ac8f None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:42.200757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978e963-55d7-4c95-9aab-19a04ea0bf4f None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:42.201076 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978e963-55d7-4c95-9aab-19a04ea0bf4f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:42.201305 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978e963-55d7-4c95-9aab-19a04ea0bf4f None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:42.202659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 139/272] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:42.208141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fda2a-6a2a-4243-8545-6de3223b5e5d None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:42.208451 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fda2a-6a2a-4243-8545-6de3223b5e5d None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:42.208687 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0fda2a-6a2a-4243-8545-6de3223b5e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:42.239509 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-de0fda2a-6a2a-4243-8545-6de3223b5e5d None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:42.241922 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-de0fda2a-6a2a-4243-8545-6de3223b5e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:42.307666 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c21a6d-f790-4577-873f-3044a264ac8f None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:42.314523 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:42.327377 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 140/273] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:42.328593 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02692d8f-48c1-46fc-9cef-1024d2b245c6 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:42.328880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02692d8f-48c1-46fc-9cef-1024d2b245c6 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:42.329154 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02692d8f-48c1-46fc-9cef-1024d2b245c6 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:42.330449 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 141/274] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:42.338593 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf2267-d0f1-4004-b28c-4cb20792c1a8 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:42.338878 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf2267-d0f1-4004-b28c-4cb20792c1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:42.339190 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bf2267-d0f1-4004-b28c-4cb20792c1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:42.344555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:42.363834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32646ec7-0973-4c56-8591-ab80499ebace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:42.364813 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32646ec7-0973-4c56-8591-ab80499ebace None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:42.365286 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32646ec7-0973-4c56-8591-ab80499ebace None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:42.365834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32646ec7-0973-4c56-8591-ab80499ebace None admin] PATH_INFO: `/v3/roles/member` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:42.375726 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-a4bf2267-d0f1-4004-b28c-4cb20792c1a8 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:42.377118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a4bf2267-d0f1-4004-b28c-4cb20792c1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:42.377796 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32646ec7-0973-4c56-8591-ab80499ebace None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:42.383312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-32646ec7-0973-4c56-8591-ab80499ebace None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:42.383312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:42.383312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:42.383312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:42.383312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:42.383312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:42.383312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:42.383312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:42.383312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:42.383312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:42.383312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:42.383312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:42.383312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:42.383312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:42.383312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:42.383312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:42.384163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:42.384163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:42.384163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:42.384163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:42.384163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:42.384163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:42.384163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:42.384163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:42.384163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:42.384163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:42.384163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:42.384163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:42.384163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:42.384163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:42.384163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:42.384163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:42.384942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:42.384942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:42.384942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:42.384942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:42.384942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:42.384942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:42.384942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:42.384942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:42.384942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:42.384942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:42.384942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:42.384942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:42.387703 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 134/275] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:42.394980 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32646ec7-0973-4c56-8591-ab80499ebace None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:42.470672 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 142/276] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:42.526962 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32646ec7-0973-4c56-8591-ab80499ebace None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:42.563861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69217be-3d17-4641-ba00-520ad83a3432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:42.564475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69217be-3d17-4641-ba00-520ad83a3432 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:42.565041 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69217be-3d17-4641-ba00-520ad83a3432 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:42.565315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69217be-3d17-4641-ba00-520ad83a3432 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:42.565980 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69217be-3d17-4641-ba00-520ad83a3432 None admin] RBAC: Adding query filter params (name=member) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:42.566286 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69217be-3d17-4641-ba00-520ad83a3432 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:42.567752 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69217be-3d17-4641-ba00-520ad83a3432 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:42.576304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 135/277] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:42.581771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf2267-d0f1-4004-b28c-4cb20792c1a8 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:42.618494 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69217be-3d17-4641-ba00-520ad83a3432 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:42.683479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:42.685277 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bf2267-d0f1-4004-b28c-4cb20792c1a8 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:42.721039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:42.729423 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ac9f8-8916-4581-9f87-18173eea5403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:42.730054 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ac9f8-8916-4581-9f87-18173eea5403 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:42.730327 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ac9f8-8916-4581-9f87-18173eea5403 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:42.730596 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ac9f8-8916-4581-9f87-18173eea5403 None admin] PATH_INFO: `/v3/users/b2d95f11f2ce4e3787be502d551497d4` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:42.743846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69217be-3d17-4641-ba00-520ad83a3432 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:42.745401 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ac9f8-8916-4581-9f87-18173eea5403 None admin] RBAC: Authorizing `identity:get_user(user_id=b2d95f11f2ce4e3787be502d551497d4)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:42.747025 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ac9f8-8916-4581-9f87-18173eea5403 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:42.753675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 143/278] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:42 2022] GET /identity/v3/users/b2d95f11f2ce4e3787be502d551497d4 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:42.754940 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ac9f8-8916-4581-9f87-18173eea5403 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:42.785764 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a1ff3-e7a1-4d21-bf2d-aab73e1c2bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:42.786395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a1ff3-e7a1-4d21-bf2d-aab73e1c2bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:42.786685 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a1ff3-e7a1-4d21-bf2d-aab73e1c2bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:42.786943 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a1ff3-e7a1-4d21-bf2d-aab73e1c2bf6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:42.788678 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a1ff3-e7a1-4d21-bf2d-aab73e1c2bf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:42.790116 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a1ff3-e7a1-4d21-bf2d-aab73e1c2bf6 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:42.791706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 136/279] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:55:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:42.792461 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a1ff3-e7a1-4d21-bf2d-aab73e1c2bf6 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:42.866964 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ac9f8-8916-4581-9f87-18173eea5403 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:42.895160 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a1ff3-e7a1-4d21-bf2d-aab73e1c2bf6 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:42.901698 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75841ca-f90b-4677-b24d-6ebdf7395ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:42.902300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75841ca-f90b-4677-b24d-6ebdf7395ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:42.902580 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75841ca-f90b-4677-b24d-6ebdf7395ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:42.902845 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75841ca-f90b-4677-b24d-6ebdf7395ca6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:42.903599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75841ca-f90b-4677-b24d-6ebdf7395ca6 None admin] RBAC: Adding query filter params (role.id=08c2a54a85ef4bb9892a5769ef8a5d95, user.id=b2d95f11f2ce4e3787be502d551497d4, scope.system=all) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:42.903906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75841ca-f90b-4677-b24d-6ebdf7395ca6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:42.905335 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75841ca-f90b-4677-b24d-6ebdf7395ca6 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:42.909438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:42.910765 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 144/280] 10.222.0.36 () {56 vars in 1308 bytes} [Fri Sep 2 08:55:42 2022] GET /identity/v3/role_assignments?user.id=b2d95f11f2ce4e3787be502d551497d4&scope.system=all&role.id=08c2a54a85ef4bb9892a5769ef8a5d95 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:42.911413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75841ca-f90b-4677-b24d-6ebdf7395ca6 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:42.940916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd941aec-f9ce-4d4b-a945-b8e549576179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:42.941972 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd941aec-f9ce-4d4b-a945-b8e549576179 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:42.942501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd941aec-f9ce-4d4b-a945-b8e549576179 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:42.943143 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd941aec-f9ce-4d4b-a945-b8e549576179 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:42.955874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd941aec-f9ce-4d4b-a945-b8e549576179 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:42.961779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-dd941aec-f9ce-4d4b-a945-b8e549576179 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 08:55:42.961779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:42.961779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:42.961779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:42.961779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:42.961779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:42.961779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:42.961779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:42.961779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:42.961779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:42.961779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:42.961779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:42.961779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:42.961779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:42.961779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:42.961779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:42.962783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:42.962783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:42.962783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:42.962783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:42.962783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:42.962783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:42.962783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:42.962783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:42.962783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:42.962783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:42.962783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:42.962783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:42.962783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:42.962783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:42.962783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:42.962783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:42.963645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:42.963645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:42.963645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:42.963645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:42.963645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:42.963645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:42.963645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:42.963645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:42.963645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:42.963645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:42.963645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 08:55:42.963645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:42.963645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 137/281] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:42.969331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffc74e-f958-4699-8c30-c53e36ebe76c None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:42.969616 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffc74e-f958-4699-8c30-c53e36ebe76c None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:42.969889 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ffc74e-f958-4699-8c30-c53e36ebe76c None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:42.971351 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 138/282] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:42.972109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ffc74e-f958-4699-8c30-c53e36ebe76c None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:42.983686 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:43.040783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75841ca-f90b-4677-b24d-6ebdf7395ca6 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:43.080297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131ff49-85a8-4603-a93b-cb6008344bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:43.081230 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131ff49-85a8-4603-a93b-cb6008344bde None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:43.081511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131ff49-85a8-4603-a93b-cb6008344bde None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:43.081779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131ff49-85a8-4603-a93b-cb6008344bde None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:43.082593 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131ff49-85a8-4603-a93b-cb6008344bde None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:43.085270 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131ff49-85a8-4603-a93b-cb6008344bde None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:43.086846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ffc74e-f958-4699-8c30-c53e36ebe76c None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:43.114095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 145/283] 10.222.0.36 () {60 vars in 1132 bytes} [Fri Sep 2 08:55:42 2022] POST /identity/v3/users => generated 293 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:55:43.115266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf24815-ea69-429b-a988-82f0f25462f9 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:43.115624 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf24815-ea69-429b-a988-82f0f25462f9 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:43.115905 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf24815-ea69-429b-a988-82f0f25462f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:43.135384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b322b-cbba-44b9-ac01-f4833f828642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:43.136585 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b322b-cbba-44b9-ac01-f4833f828642 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:43.137120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b322b-cbba-44b9-ac01-f4833f828642 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:43.137670 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b322b-cbba-44b9-ac01-f4833f828642 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:43.138002 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-edf24815-ea69-429b-a988-82f0f25462f9 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:43.138852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b322b-cbba-44b9-ac01-f4833f828642 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:43.139278 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-edf24815-ea69-429b-a988-82f0f25462f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:43.139709 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b322b-cbba-44b9-ac01-f4833f828642 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:43.141819 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b322b-cbba-44b9-ac01-f4833f828642 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:43.156495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 139/284] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:43.162280 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b322b-cbba-44b9-ac01-f4833f828642 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:43.202762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:43.224892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 146/285] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:43.276022 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b322b-cbba-44b9-ac01-f4833f828642 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:43.312833 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a5841-0fc4-44ae-8d1b-104672d3752a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:43.313518 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a5841-0fc4-44ae-8d1b-104672d3752a None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:43.313803 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a5841-0fc4-44ae-8d1b-104672d3752a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:43.314076 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a5841-0fc4-44ae-8d1b-104672d3752a None admin] PATH_INFO: `/v3/users/39c417d520664dbf9f4b95841e4903b0` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:43.315247 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a5841-0fc4-44ae-8d1b-104672d3752a None admin] RBAC: Authorizing `identity:get_user(user_id=39c417d520664dbf9f4b95841e4903b0)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:43.316699 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a5841-0fc4-44ae-8d1b-104672d3752a None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:43.318575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 140/286] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:43 2022] GET /identity/v3/users/39c417d520664dbf9f4b95841e4903b0 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:43.324176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf24815-ea69-429b-a988-82f0f25462f9 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:43.369526 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:43.434235 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf24815-ea69-429b-a988-82f0f25462f9 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:43.473042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e7644-61b9-4e34-9a35-76cfd50339cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:43.473689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e7644-61b9-4e34-9a35-76cfd50339cd None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:43.473969 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e7644-61b9-4e34-9a35-76cfd50339cd None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:43.474242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296e7644-61b9-4e34-9a35-76cfd50339cd None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:43.475434 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e7644-61b9-4e34-9a35-76cfd50339cd None admin] RBAC: Authorizing `identity:get_project(project_id=78733ddb1b714faf83957113a15c51bc)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:43.476882 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296e7644-61b9-4e34-9a35-76cfd50339cd None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:43.478503 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 147/287] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:43 2022] GET /identity/v3/projects/78733ddb1b714faf83957113a15c51bc => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:43.484316 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a5841-0fc4-44ae-8d1b-104672d3752a None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:43.496019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e7644-61b9-4e34-9a35-76cfd50339cd None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:43.594525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a5841-0fc4-44ae-8d1b-104672d3752a None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:43.597104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:43.613537 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296e7644-61b9-4e34-9a35-76cfd50339cd None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:43.624179 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:43.633384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e77487-bb19-40fc-a29b-81e6c008abbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:43.634114 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e77487-bb19-40fc-a29b-81e6c008abbb None admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:43.634402 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e77487-bb19-40fc-a29b-81e6c008abbb None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:43.634679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e77487-bb19-40fc-a29b-81e6c008abbb None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc/users/39c417d520664dbf9f4b95841e4903b0/roles/b1bd3fa77e3d4c83aa57374cb35fd21f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:43.636661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e77487-bb19-40fc-a29b-81e6c008abbb None admin] RBAC: Authorizing `identity:create_grant(project_id=78733ddb1b714faf83957113a15c51bc, user_id=39c417d520664dbf9f4b95841e4903b0, role_id=b1bd3fa77e3d4c83aa57374cb35fd21f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:43.638135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e77487-bb19-40fc-a29b-81e6c008abbb None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:43.659026 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 141/288] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:55:43 2022] PUT /identity/v3/projects/78733ddb1b714faf83957113a15c51bc/users/39c417d520664dbf9f4b95841e4903b0/roles/b1bd3fa77e3d4c83aa57374cb35fd21f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:55:43.665606 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa75a07-f00d-4f7e-946a-6b5e7569d330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:43.666702 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa75a07-f00d-4f7e-946a-6b5e7569d330 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:43.667286 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa75a07-f00d-4f7e-946a-6b5e7569d330 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:43.667816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa75a07-f00d-4f7e-946a-6b5e7569d330 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:43.681521 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa75a07-f00d-4f7e-946a-6b5e7569d330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:43.687563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-bfa75a07-f00d-4f7e-946a-6b5e7569d330 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:43.687563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:43.687563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:43.687563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:43.687563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:43.687563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:43.687563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:43.687563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:43.687563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:43.687563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:43.687563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:43.687563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:43.687563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:43.687563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:43.687563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:43.687563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:43.688934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:43.688934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:43.688934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:43.688934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:43.688934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:43.688934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:43.688934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:43.688934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:43.688934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:43.688934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:43.688934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:43.688934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:43.688934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:43.688934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:43.688934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:43.688934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:43.689830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:43.689830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:43.689830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:43.689830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:43.689830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:43.689830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:43.689830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:43.689830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:43.689830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:43.689830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:43.689830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:43.689830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:43.692867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 148/289] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:43.699736 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e77487-bb19-40fc-a29b-81e6c008abbb None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:43.846746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e77487-bb19-40fc-a29b-81e6c008abbb None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:43.895313 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b24789-6bf7-4520-b7a5-0e434453f45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:43.895926 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b24789-6bf7-4520-b7a5-0e434453f45b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:43.896200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b24789-6bf7-4520-b7a5-0e434453f45b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:43.896467 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b24789-6bf7-4520-b7a5-0e434453f45b None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:43.897175 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b24789-6bf7-4520-b7a5-0e434453f45b None admin] RBAC: Adding query filter params (name=member) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:43.897474 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b24789-6bf7-4520-b7a5-0e434453f45b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:43.898844 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b24789-6bf7-4520-b7a5-0e434453f45b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:43.911301 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 142/290] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:43.916623 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa75a07-f00d-4f7e-946a-6b5e7569d330 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:44.035513 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa75a07-f00d-4f7e-946a-6b5e7569d330 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:44.089760 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111f22f-02e8-4076-adbe-a3a71a1d288a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:44.090761 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111f22f-02e8-4076-adbe-a3a71a1d288a None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:44.091296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111f22f-02e8-4076-adbe-a3a71a1d288a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:44.091765 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111f22f-02e8-4076-adbe-a3a71a1d288a None admin] PATH_INFO: `/v3/users/680233e444574c71bfdfeff4e94f799d` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:44.093129 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111f22f-02e8-4076-adbe-a3a71a1d288a None admin] RBAC: Authorizing `identity:get_user(user_id=680233e444574c71bfdfeff4e94f799d)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:44.094862 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111f22f-02e8-4076-adbe-a3a71a1d288a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:44.096754 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 149/291] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:43 2022] GET /identity/v3/users/680233e444574c71bfdfeff4e94f799d => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:44.102738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b24789-6bf7-4520-b7a5-0e434453f45b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:44.143701 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:44.230699 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b24789-6bf7-4520-b7a5-0e434453f45b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:44.277260 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517df5c1-7057-4b05-9ea6-389f91f1ce58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:44.278208 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517df5c1-7057-4b05-9ea6-389f91f1ce58 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:44.278755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517df5c1-7057-4b05-9ea6-389f91f1ce58 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:44.279256 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517df5c1-7057-4b05-9ea6-389f91f1ce58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:44.353832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517df5c1-7057-4b05-9ea6-389f91f1ce58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:44.355694 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517df5c1-7057-4b05-9ea6-389f91f1ce58 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:44.399483 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:44.477197 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 143/292] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:44.482765 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111f22f-02e8-4076-adbe-a3a71a1d288a None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:44.541787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:44.625649 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111f22f-02e8-4076-adbe-a3a71a1d288a None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:44.669231 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525cacf-caa3-4b21-a99e-c6c05dffbb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:44.669874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525cacf-caa3-4b21-a99e-c6c05dffbb26 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:44.670156 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525cacf-caa3-4b21-a99e-c6c05dffbb26 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:44.670428 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525cacf-caa3-4b21-a99e-c6c05dffbb26 None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:44.671573 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525cacf-caa3-4b21-a99e-c6c05dffbb26 None admin] RBAC: Authorizing `identity:get_project(project_id=78733ddb1b714faf83957113a15c51bc)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:44.673074 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525cacf-caa3-4b21-a99e-c6c05dffbb26 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:44.675011 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 150/293] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:44 2022] GET /identity/v3/projects/78733ddb1b714faf83957113a15c51bc => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:44.681215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517df5c1-7057-4b05-9ea6-389f91f1ce58 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:44.827506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517df5c1-7057-4b05-9ea6-389f91f1ce58 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:44.858400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:44.868568 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9fa5b3-4184-4fda-891d-3bd0d0c24f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:44.869450 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fa5b3-4184-4fda-891d-3bd0d0c24f72 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:44.869942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fa5b3-4184-4fda-891d-3bd0d0c24f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:44.870405 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9fa5b3-4184-4fda-891d-3bd0d0c24f72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:44.871347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fa5b3-4184-4fda-891d-3bd0d0c24f72 None admin] RBAC: Adding query filter params (role.id=08c2a54a85ef4bb9892a5769ef8a5d95, scope.project.id=78733ddb1b714faf83957113a15c51bc, user.id=680233e444574c71bfdfeff4e94f799d) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:44.871848 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fa5b3-4184-4fda-891d-3bd0d0c24f72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:44.873473 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9fa5b3-4184-4fda-891d-3bd0d0c24f72 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:44.879933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 144/294] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:55:44 2022] GET /identity/v3/role_assignments?user.id=680233e444574c71bfdfeff4e94f799d&scope.project.id=78733ddb1b714faf83957113a15c51bc&role.id=08c2a54a85ef4bb9892a5769ef8a5d95 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:45.312357 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098b164-86b2-4d99-9c2f-ed5fd6520ac8 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:45.312842 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098b164-86b2-4d99-9c2f-ed5fd6520ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:45.312842 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098b164-86b2-4d99-9c2f-ed5fd6520ac8 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:45.314455 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 151/295] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:45.319933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c9ab7-ea98-4c1d-9d01-72230876c05c None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:45.320217 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c9ab7-ea98-4c1d-9d01-72230876c05c None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:45.320484 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c9ab7-ea98-4c1d-9d01-72230876c05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:45.344137 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-032c9ab7-ea98-4c1d-9d01-72230876c05c None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:45.345323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-032c9ab7-ea98-4c1d-9d01-72230876c05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:45.423039 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 145/296] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:45.543202 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3d009-0638-47ca-bece-ab1d7cca7104 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:45.543514 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3d009-0638-47ca-bece-ab1d7cca7104 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:45.543755 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3d009-0638-47ca-bece-ab1d7cca7104 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:45.545354 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 152/297] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:45.550428 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ddbf4-08b0-4250-b27f-e6a4da8b9e3a None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:45.550722 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ddbf4-08b0-4250-b27f-e6a4da8b9e3a None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:45.551002 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ddbf4-08b0-4250-b27f-e6a4da8b9e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:45.572460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-554ddbf4-08b0-4250-b27f-e6a4da8b9e3a None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:45.573574 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-554ddbf4-08b0-4250-b27f-e6a4da8b9e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:45.640375 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 146/298] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:45.669887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87c046-875a-4a74-9b05-e5720e8e7daa None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:45.670165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87c046-875a-4a74-9b05-e5720e8e7daa None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:45.670432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87c046-875a-4a74-9b05-e5720e8e7daa None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:45.672124 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 153/299] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:45.677203 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda4a41-8d19-4525-b5a0-98bc7c7da097 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:45.677496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda4a41-8d19-4525-b5a0-98bc7c7da097 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:45.677757 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda4a41-8d19-4525-b5a0-98bc7c7da097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:45.696916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-5dda4a41-8d19-4525-b5a0-98bc7c7da097 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:45.698079 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-5dda4a41-8d19-4525-b5a0-98bc7c7da097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:45.728529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87c046-875a-4a74-9b05-e5720e8e7daa None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:45.747277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:45.769434 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 147/300] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:45.841216 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87c046-875a-4a74-9b05-e5720e8e7daa None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:45.842582 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:45.877539 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a55af4-163a-480e-8caa-4b0087e808da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:45.878167 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a55af4-163a-480e-8caa-4b0087e808da None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:45.878443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a55af4-163a-480e-8caa-4b0087e808da None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:45.878708 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a55af4-163a-480e-8caa-4b0087e808da None admin] PATH_INFO: `/v3/roles/member` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:45.889117 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a55af4-163a-480e-8caa-4b0087e808da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:45.893933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-f9a55af4-163a-480e-8caa-4b0087e808da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:45.893933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:45.893933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:45.893933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:45.893933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:45.893933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:45.893933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:45.893933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:45.893933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:45.893933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:45.893933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:45.893933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:45.893933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:45.893933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:45.893933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:45.893933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:45.894816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:45.894816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:45.894816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:45.894816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:45.894816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:45.894816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:45.894816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:45.894816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:45.894816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:45.894816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:45.894816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:45.894816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:45.894816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:45.894816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:45.894816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:45.894816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:45.895646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:45.895646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:45.895646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:45.895646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:45.895646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:45.895646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:45.895646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:45.895646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:45.895646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:45.895646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:45.895646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:45.895646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:45.895646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 154/301] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:45.901285 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda4a41-8d19-4525-b5a0-98bc7c7da097 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:45.915518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a55af4-163a-480e-8caa-4b0087e808da None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:46.024692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a55af4-163a-480e-8caa-4b0087e808da None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:46.025329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda4a41-8d19-4525-b5a0-98bc7c7da097 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:46.061703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c757361-a1c7-42d7-acb6-366c02d46ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:46.062423 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c757361-a1c7-42d7-acb6-366c02d46ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:46.062699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c757361-a1c7-42d7-acb6-366c02d46ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:46.062966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c757361-a1c7-42d7-acb6-366c02d46ff6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:46.068671 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dcbe2-63f7-49f7-bc1f-f37376e4c938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:46.069250 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dcbe2-63f7-49f7-bc1f-f37376e4c938 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:46.069523 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dcbe2-63f7-49f7-bc1f-f37376e4c938 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:46.069789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dcbe2-63f7-49f7-bc1f-f37376e4c938 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:46.070449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874dcbe2-63f7-49f7-bc1f-f37376e4c938 None admin] RBAC: Adding query filter params (name=member) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:46.070747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874dcbe2-63f7-49f7-bc1f-f37376e4c938 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:46.072140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874dcbe2-63f7-49f7-bc1f-f37376e4c938 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:46.073859 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c757361-a1c7-42d7-acb6-366c02d46ff6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:46.079398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-3c757361-a1c7-42d7-acb6-366c02d46ff6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 08:55:46.079398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:46.079398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:46.079398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:46.079398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:46.079398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:46.079398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:46.079398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:46.079398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:46.079398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:46.079398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:46.079398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:46.079398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:46.079398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:46.079398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:46.079398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:46.080294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:46.080294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:46.080294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:46.080294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:46.080294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:46.080294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:46.080294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:46.080294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:46.080294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:46.080294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:46.080294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:46.080294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:46.080294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:46.080294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:46.080294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:46.080294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:46.084038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:46.084038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:46.084038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:46.084038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:46.084038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:46.084038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:46.084038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:46.084038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:46.084038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:46.084038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:46.084038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 08:55:46.084038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:46.084038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 155/302] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:46.084038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c757361-a1c7-42d7-acb6-366c02d46ff6 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:46.085850 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 148/303] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:46.086756 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dcbe2-63f7-49f7-bc1f-f37376e4c938 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:46.111407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:46.131506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:46.189804 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dcbe2-63f7-49f7-bc1f-f37376e4c938 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:46.192238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c757361-a1c7-42d7-acb6-366c02d46ff6 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:46.222286 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9220b38e-5b0f-4ebb-9664-327d2390a2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:46.222891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220b38e-5b0f-4ebb-9664-327d2390a2c2 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:46.223197 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220b38e-5b0f-4ebb-9664-327d2390a2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:46.223471 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9220b38e-5b0f-4ebb-9664-327d2390a2c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:46.224124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220b38e-5b0f-4ebb-9664-327d2390a2c2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:46.224424 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220b38e-5b0f-4ebb-9664-327d2390a2c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:46.225973 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ada19-7895-41a2-b61f-b82cc28d166b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:46.226357 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9220b38e-5b0f-4ebb-9664-327d2390a2c2 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:46.226627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ada19-7895-41a2-b61f-b82cc28d166b None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:46.226906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ada19-7895-41a2-b61f-b82cc28d166b None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:46.227212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ada19-7895-41a2-b61f-b82cc28d166b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:46.228300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ada19-7895-41a2-b61f-b82cc28d166b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:46.229717 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1ada19-7895-41a2-b61f-b82cc28d166b None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:46.231310 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 156/304] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:55:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:46.232060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ada19-7895-41a2-b61f-b82cc28d166b None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:46.234491 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 149/305] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:46.236061 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423eba66-7332-4911-b577-f8984f498ecf None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:46.237020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423eba66-7332-4911-b577-f8984f498ecf None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:46.237020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423eba66-7332-4911-b577-f8984f498ecf None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:46.239396 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 150/306] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:46.239396 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423eba66-7332-4911-b577-f8984f498ecf None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:46.320999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:46.330647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ada19-7895-41a2-b61f-b82cc28d166b None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:46.339695 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423eba66-7332-4911-b577-f8984f498ecf None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:46.342660 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:46.367144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd40d8-13c0-4aca-8fe8-480c7932e726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:46.367710 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd40d8-13c0-4aca-8fe8-480c7932e726 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:46.367989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd40d8-13c0-4aca-8fe8-480c7932e726 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:46.368262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd40d8-13c0-4aca-8fe8-480c7932e726 None admin] PATH_INFO: `/v3/users/b2d95f11f2ce4e3787be502d551497d4` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:46.369356 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd40d8-13c0-4aca-8fe8-480c7932e726 None admin] RBAC: Authorizing `identity:get_user(user_id=b2d95f11f2ce4e3787be502d551497d4)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:46.370777 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd40d8-13c0-4aca-8fe8-480c7932e726 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:46.372516 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 157/307] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:46 2022] GET /identity/v3/users/b2d95f11f2ce4e3787be502d551497d4 => generated 293 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:46.373167 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd40d8-13c0-4aca-8fe8-480c7932e726 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:46.379452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18d22b-fc4a-4807-a95d-70ff45fa332d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:46.380064 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18d22b-fc4a-4807-a95d-70ff45fa332d None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:46.380346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18d22b-fc4a-4807-a95d-70ff45fa332d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:46.380617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18d22b-fc4a-4807-a95d-70ff45fa332d None admin] PATH_INFO: `/v3/groups` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:46.381502 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18d22b-fc4a-4807-a95d-70ff45fa332d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:46.382933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18d22b-fc4a-4807-a95d-70ff45fa332d None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:46.399551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 151/308] 10.222.0.36 () {60 vars in 1134 bytes} [Fri Sep 2 08:55:46 2022] POST /identity/v3/groups => generated 229 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:55:46.400729 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e622e44-a56d-44d1-b2fc-26a1a5e57606 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:46.401010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e622e44-a56d-44d1-b2fc-26a1a5e57606 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:46.401292 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e622e44-a56d-44d1-b2fc-26a1a5e57606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:46.422107 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-1e622e44-a56d-44d1-b2fc-26a1a5e57606 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:46.423402 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1e622e44-a56d-44d1-b2fc-26a1a5e57606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:46.491661 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd40d8-13c0-4aca-8fe8-480c7932e726 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:46.494635 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 152/309] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:46.495581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e622e44-a56d-44d1-b2fc-26a1a5e57606 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:46.535403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a701a4f-cb80-4b43-8bb4-215f9118c707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:46.536179 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a701a4f-cb80-4b43-8bb4-215f9118c707 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:46.536521 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a701a4f-cb80-4b43-8bb4-215f9118c707 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:46.536860 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a701a4f-cb80-4b43-8bb4-215f9118c707 None admin] PATH_INFO: `/v3/users/2301e4f1cc2d4728b95a1e44f766583f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:46.553859 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a701a4f-cb80-4b43-8bb4-215f9118c707 None admin] RBAC: Authorizing `identity:get_user(user_id=2301e4f1cc2d4728b95a1e44f766583f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:46.555454 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a701a4f-cb80-4b43-8bb4-215f9118c707 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:46.557222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 158/310] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:46 2022] GET /identity/v3/users/2301e4f1cc2d4728b95a1e44f766583f => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:46.572578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a701a4f-cb80-4b43-8bb4-215f9118c707 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:46.600218 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:46.622570 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:46.633602 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e622e44-a56d-44d1-b2fc-26a1a5e57606 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:46.678611 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826acbbc-25b2-4e0c-830f-02d705adc7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:46.679446 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826acbbc-25b2-4e0c-830f-02d705adc7fd None admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:46.679788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826acbbc-25b2-4e0c-830f-02d705adc7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:46.680124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826acbbc-25b2-4e0c-830f-02d705adc7fd None admin] PATH_INFO: `/v3/system/users/b2d95f11f2ce4e3787be502d551497d4/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:46.682303 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826acbbc-25b2-4e0c-830f-02d705adc7fd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b2d95f11f2ce4e3787be502d551497d4, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:46.687107 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826acbbc-25b2-4e0c-830f-02d705adc7fd None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:46.709751 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 153/311] 10.222.0.36 () {58 vars in 1334 bytes} [Fri Sep 2 08:55:46 2022] PUT /identity/v3/system/users/b2d95f11f2ce4e3787be502d551497d4/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:55:46.750113 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a701a4f-cb80-4b43-8bb4-215f9118c707 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:46.792004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826acbbc-25b2-4e0c-830f-02d705adc7fd None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:46.792961 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa9d14-c902-4016-a502-fd10db326564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:46.793573 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa9d14-c902-4016-a502-fd10db326564 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:46.793845 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa9d14-c902-4016-a502-fd10db326564 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:46.794111 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa9d14-c902-4016-a502-fd10db326564 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:46.794841 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa9d14-c902-4016-a502-fd10db326564 None admin] RBAC: Adding query filter params (role.id=b1bd3fa77e3d4c83aa57374cb35fd21f, user.id=2301e4f1cc2d4728b95a1e44f766583f, scope.system=all) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:46.795181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa9d14-c902-4016-a502-fd10db326564 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:46.796640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa9d14-c902-4016-a502-fd10db326564 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:46.803598 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 159/312] 10.222.0.36 () {56 vars in 1308 bytes} [Fri Sep 2 08:55:46 2022] GET /identity/v3/role_assignments?user.id=2301e4f1cc2d4728b95a1e44f766583f&scope.system=all&role.id=b1bd3fa77e3d4c83aa57374cb35fd21f => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:46.964578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826acbbc-25b2-4e0c-830f-02d705adc7fd None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:46.999853 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe3b18-6c6d-4f5a-9462-71ed0e0dfbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:47.000490 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe3b18-6c6d-4f5a-9462-71ed0e0dfbb4 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:47.000766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe3b18-6c6d-4f5a-9462-71ed0e0dfbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:47.001037 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe3b18-6c6d-4f5a-9462-71ed0e0dfbb4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:47.014847 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe3b18-6c6d-4f5a-9462-71ed0e0dfbb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:47.020186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-36fe3b18-6c6d-4f5a-9462-71ed0e0dfbb4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 08:55:47.020186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:47.020186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:47.020186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:47.020186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:47.020186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:47.020186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:47.020186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:47.020186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:47.020186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:47.020186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:47.020186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:47.020186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:47.020186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:47.020186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:47.020186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:47.021074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:47.021074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:47.021074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:47.021074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:47.021074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:47.021074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:47.021074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:47.021074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:47.021074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:47.021074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:47.021074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:47.021074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:47.021074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:47.021074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:47.021074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:47.021074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:47.021914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:47.021914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:47.021914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:47.021914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:47.021914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:47.021914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:47.021914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:47.021914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:47.021914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:47.021914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:47.021914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 08:55:47.021914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:47.021914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 154/313] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:55:47.028894 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa9d14-c902-4016-a502-fd10db326564 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:47.189294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:47.214555 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa9d14-c902-4016-a502-fd10db326564 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:47.251661 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d1e6e-3557-476a-a2e7-43a674911317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:47.252617 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d1e6e-3557-476a-a2e7-43a674911317 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:47.253099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d1e6e-3557-476a-a2e7-43a674911317 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:47.253573 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d1e6e-3557-476a-a2e7-43a674911317 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:47.254426 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d1e6e-3557-476a-a2e7-43a674911317 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:47.254934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d1e6e-3557-476a-a2e7-43a674911317 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:47.256589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d1e6e-3557-476a-a2e7-43a674911317 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:47.265989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 160/314] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:47.271899 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe3b18-6c6d-4f5a-9462-71ed0e0dfbb4 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:47.291882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:47.376500 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe3b18-6c6d-4f5a-9462-71ed0e0dfbb4 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:47.411479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979f9c4-fdd3-4167-838e-a43548952489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:47.412662 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979f9c4-fdd3-4167-838e-a43548952489 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:47.412932 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979f9c4-fdd3-4167-838e-a43548952489 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:47.413201 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979f9c4-fdd3-4167-838e-a43548952489 None admin] PATH_INFO: `/v3/users/39c417d520664dbf9f4b95841e4903b0` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:47.414383 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979f9c4-fdd3-4167-838e-a43548952489 None admin] RBAC: Authorizing `identity:get_user(user_id=39c417d520664dbf9f4b95841e4903b0)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:47.415865 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979f9c4-fdd3-4167-838e-a43548952489 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:47.417664 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 155/315] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:47 2022] GET /identity/v3/users/39c417d520664dbf9f4b95841e4903b0 => generated 297 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:47.423413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d1e6e-3557-476a-a2e7-43a674911317 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:47.531157 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d1e6e-3557-476a-a2e7-43a674911317 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:47.566590 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b613ea3-f680-4dcd-9d7a-72571fea4f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:47.567405 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b613ea3-f680-4dcd-9d7a-72571fea4f49 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:47.567827 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b613ea3-f680-4dcd-9d7a-72571fea4f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:47.568239 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b613ea3-f680-4dcd-9d7a-72571fea4f49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:47.580386 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:47.637640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b613ea3-f680-4dcd-9d7a-72571fea4f49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:47.639370 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b613ea3-f680-4dcd-9d7a-72571fea4f49 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:47.742430 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 161/316] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:47.747977 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979f9c4-fdd3-4167-838e-a43548952489 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:47.844435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:47.861387 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979f9c4-fdd3-4167-838e-a43548952489 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:47.894922 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a70c3-9743-4636-93dc-ac4a34e2a855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:47.895612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a70c3-9743-4636-93dc-ac4a34e2a855 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:47.895887 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a70c3-9743-4636-93dc-ac4a34e2a855 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:47.896155 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a70c3-9743-4636-93dc-ac4a34e2a855 None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:47.897899 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a70c3-9743-4636-93dc-ac4a34e2a855 None admin] RBAC: Authorizing `identity:get_project(project_id=78733ddb1b714faf83957113a15c51bc)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:47.899399 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3a70c3-9743-4636-93dc-ac4a34e2a855 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:47.901124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 156/317] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:47 2022] GET /identity/v3/projects/78733ddb1b714faf83957113a15c51bc => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:47.907144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b613ea3-f680-4dcd-9d7a-72571fea4f49 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:47.951559 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:48.012322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b613ea3-f680-4dcd-9d7a-72571fea4f49 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:48.046896 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f888deb-d6b7-45f6-a2f4-47f2c8966e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:48.047769 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f888deb-d6b7-45f6-a2f4-47f2c8966e01 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:48.048189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f888deb-d6b7-45f6-a2f4-47f2c8966e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:48.048602 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f888deb-d6b7-45f6-a2f4-47f2c8966e01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:48.049483 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f888deb-d6b7-45f6-a2f4-47f2c8966e01 None admin] RBAC: Adding query filter params (role.id=b1bd3fa77e3d4c83aa57374cb35fd21f, scope.project.id=78733ddb1b714faf83957113a15c51bc, user.id=39c417d520664dbf9f4b95841e4903b0) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:48.049945 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f888deb-d6b7-45f6-a2f4-47f2c8966e01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:48.051554 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f888deb-d6b7-45f6-a2f4-47f2c8966e01 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:48.057831 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 162/318] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:55:47 2022] GET /identity/v3/role_assignments?user.id=39c417d520664dbf9f4b95841e4903b0&scope.project.id=78733ddb1b714faf83957113a15c51bc&role.id=b1bd3fa77e3d4c83aa57374cb35fd21f => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:48.591013 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731fa61-dae2-4114-b5f7-128e3b3c3183 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:48.591656 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731fa61-dae2-4114-b5f7-128e3b3c3183 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:48.591656 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731fa61-dae2-4114-b5f7-128e3b3c3183 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:48.593069 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 157/319] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:48.598161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68158072-1d4b-4442-804f-f12d95ec5569 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:48.598557 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68158072-1d4b-4442-804f-f12d95ec5569 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:48.598752 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68158072-1d4b-4442-804f-f12d95ec5569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:48.616920 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-68158072-1d4b-4442-804f-f12d95ec5569 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:48.618017 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-68158072-1d4b-4442-804f-f12d95ec5569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:48.684921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 163/320] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:48.753158 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc28ab-5a7c-4959-b97e-edb7f51ac214 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:48.753483 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc28ab-5a7c-4959-b97e-edb7f51ac214 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:48.753708 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc28ab-5a7c-4959-b97e-edb7f51ac214 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:48.755107 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 158/321] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:48.760121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc0a0e-c3dd-4448-b748-b02ebe597eba None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:48.760414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc0a0e-c3dd-4448-b748-b02ebe597eba None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:48.760690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc0a0e-c3dd-4448-b748-b02ebe597eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:48.778451 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-1bcc0a0e-c3dd-4448-b748-b02ebe597eba None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:48.779644 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-1bcc0a0e-c3dd-4448-b748-b02ebe597eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:48.809163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:48.821868 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb88666-5b20-4e8b-924c-50d74a84c75c None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:48.822146 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb88666-5b20-4e8b-924c-50d74a84c75c None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:48.822418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb88666-5b20-4e8b-924c-50d74a84c75c None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:48.823915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 159/322] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:55:48.828480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fe979-62e4-4f4e-8015-e1adc676beb0 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:48.828762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fe979-62e4-4f4e-8015-e1adc676beb0 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:48.829038 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fe979-62e4-4f4e-8015-e1adc676beb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:48.848079 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-560fe979-62e4-4f4e-8015-e1adc676beb0 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:48.848658 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 164/323] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:48.849269 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-560fe979-62e4-4f4e-8015-e1adc676beb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:48.921111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 160/324] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:55:48.954352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc0a0e-c3dd-4448-b748-b02ebe597eba None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:49.052384 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc0a0e-c3dd-4448-b748-b02ebe597eba None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:49.085790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb197ce6-34d9-487a-a160-eca1bf780dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:49.086409 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb197ce6-34d9-487a-a160-eca1bf780dab None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:49.086682 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb197ce6-34d9-487a-a160-eca1bf780dab None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:49.086953 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb197ce6-34d9-487a-a160-eca1bf780dab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:49.088058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb197ce6-34d9-487a-a160-eca1bf780dab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:49.089486 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb197ce6-34d9-487a-a160-eca1bf780dab None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:49.091184 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 165/325] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:55:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:49.096638 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fe979-62e4-4f4e-8015-e1adc676beb0 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:49.115010 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb197ce6-34d9-487a-a160-eca1bf780dab None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:49.151196 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:49.185963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:49.203609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fe979-62e4-4f4e-8015-e1adc676beb0 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:49.220989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb197ce6-34d9-487a-a160-eca1bf780dab None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:49.238109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d5779-86e0-4b08-9d7a-0b87b277a91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:49.238708 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d5779-86e0-4b08-9d7a-0b87b277a91f None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:49.238985 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d5779-86e0-4b08-9d7a-0b87b277a91f None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:49.239280 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d5779-86e0-4b08-9d7a-0b87b277a91f None admin] PATH_INFO: `/v3/groups` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:49.239910 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d5779-86e0-4b08-9d7a-0b87b277a91f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:49.241401 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d5779-86e0-4b08-9d7a-0b87b277a91f None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:49.254263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b65be-a145-4484-8169-68fd934e2ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:49.254886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b65be-a145-4484-8169-68fd934e2ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:49.255190 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b65be-a145-4484-8169-68fd934e2ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:49.255462 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7b65be-a145-4484-8169-68fd934e2ee2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:49.256950 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 161/326] 10.222.0.36 () {60 vars in 1134 bytes} [Fri Sep 2 08:55:49 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:55:49.257741 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d5779-86e0-4b08-9d7a-0b87b277a91f None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:49.277368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7b65be-a145-4484-8169-68fd934e2ee2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:49.282446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-4b7b65be-a145-4484-8169-68fd934e2ee2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:49.282446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:49.282446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:49.282446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:49.282446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:49.282446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:49.282446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:49.282446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:49.282446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:49.282446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:49.282446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:49.282446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:49.282446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:49.282446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:49.282446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:49.282446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:49.283652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:49.283652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:49.283652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:49.283652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:49.283652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:49.283652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:49.283652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:49.283652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:49.283652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:49.283652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:49.283652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:49.283652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:49.283652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:49.283652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:49.283652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:49.283652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:49.284475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:49.284475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:49.284475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:49.284475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:49.284475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:49.284475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:49.284475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:49.284475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:49.284475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:49.284475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:49.284475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:49.284475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:49.284475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 166/327] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:49.290333 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b65be-a145-4484-8169-68fd934e2ee2 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:49.382695 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d5779-86e0-4b08-9d7a-0b87b277a91f None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:49.400570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:49.409649 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7b65be-a145-4484-8169-68fd934e2ee2 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:49.423332 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce6819-5ad7-4c8b-9fe0-5aed68637d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:49.423967 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce6819-5ad7-4c8b-9fe0-5aed68637d6d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:49.424244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce6819-5ad7-4c8b-9fe0-5aed68637d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:49.424600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce6819-5ad7-4c8b-9fe0-5aed68637d6d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:49.436626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce6819-5ad7-4c8b-9fe0-5aed68637d6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:49.441976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-97ce6819-5ad7-4c8b-9fe0-5aed68637d6d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 08:55:49.441976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:49.441976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:49.441976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:49.441976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:49.441976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:49.441976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:49.441976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:49.441976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:49.441976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:49.441976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:49.441976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:49.441976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:49.441976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:49.441976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:49.441976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:49.442901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:49.442901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:49.442901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:49.442901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:49.442901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:49.442901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:49.442901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:49.442901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:49.442901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:49.442901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:49.442901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:49.442901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:49.442901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:49.442901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:49.442901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:49.442901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:49.444271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:49.444271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:49.444271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:49.444271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:49.444271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:49.444271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:49.444271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:49.444271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:49.444271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:49.444271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:49.444271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 08:55:49.444271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:49.444271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 162/328] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:49.449375 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce6819-5ad7-4c8b-9fe0-5aed68637d6d None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:49.455099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38c679-4327-4864-9a24-60ef112b35e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:49.455546 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:49.455767 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38c679-4327-4864-9a24-60ef112b35e0 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:49.456847 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38c679-4327-4864-9a24-60ef112b35e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:49.456847 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38c679-4327-4864-9a24-60ef112b35e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:49.457304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38c679-4327-4864-9a24-60ef112b35e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:49.457604 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38c679-4327-4864-9a24-60ef112b35e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:49.462708 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38c679-4327-4864-9a24-60ef112b35e0 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:49.470969 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 167/329] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:49.477221 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38c679-4327-4864-9a24-60ef112b35e0 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:49.585861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce6819-5ad7-4c8b-9fe0-5aed68637d6d None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:49.600666 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38c679-4327-4864-9a24-60ef112b35e0 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:49.640175 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f02a43-039b-430a-95e2-98fc27657179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:49.640792 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f02a43-039b-430a-95e2-98fc27657179 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:49.641074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f02a43-039b-430a-95e2-98fc27657179 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:49.641350 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f02a43-039b-430a-95e2-98fc27657179 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:49.642024 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f02a43-039b-430a-95e2-98fc27657179 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:49.642331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f02a43-039b-430a-95e2-98fc27657179 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:49.643744 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f02a43-039b-430a-95e2-98fc27657179 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:49.650458 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa82d2a9-c9a4-46fe-b15a-30d6b5741480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:49.651241 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82d2a9-c9a4-46fe-b15a-30d6b5741480 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:49.651610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82d2a9-c9a4-46fe-b15a-30d6b5741480 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:49.651953 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82d2a9-c9a4-46fe-b15a-30d6b5741480 None admin] PATH_INFO: `/v3/users/b2d95f11f2ce4e3787be502d551497d4` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:49.654772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa82d2a9-c9a4-46fe-b15a-30d6b5741480 None admin] RBAC: Authorizing `identity:get_user(user_id=b2d95f11f2ce4e3787be502d551497d4)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:49.654888 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 163/330] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:49.656259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa82d2a9-c9a4-46fe-b15a-30d6b5741480 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:49.657961 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 168/331] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:49 2022] GET /identity/v3/users/b2d95f11f2ce4e3787be502d551497d4 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:49.660737 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f02a43-039b-430a-95e2-98fc27657179 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:49.666068 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa82d2a9-c9a4-46fe-b15a-30d6b5741480 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:49.695658 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:49.751520 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:49.787830 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f02a43-039b-430a-95e2-98fc27657179 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:49.788381 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa82d2a9-c9a4-46fe-b15a-30d6b5741480 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:49.825205 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1d011-7def-4025-b754-6574fc6bef1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:49.826276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1d011-7def-4025-b754-6574fc6bef1f None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:49.826696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1d011-7def-4025-b754-6574fc6bef1f None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:49.827131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1d011-7def-4025-b754-6574fc6bef1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:49.827556 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b59ad-645d-4a46-a270-8819db760b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:49.828238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1d011-7def-4025-b754-6574fc6bef1f None admin] RBAC: Adding query filter params (role.id=08c2a54a85ef4bb9892a5769ef8a5d95, user.id=b2d95f11f2ce4e3787be502d551497d4, scope.system=all) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:49.828358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b59ad-645d-4a46-a270-8819db760b7c None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:49.828619 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b59ad-645d-4a46-a270-8819db760b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:49.828964 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b59ad-645d-4a46-a270-8819db760b7c None admin] PATH_INFO: `/v3/users/2301e4f1cc2d4728b95a1e44f766583f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:49.829160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1d011-7def-4025-b754-6574fc6bef1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:49.830229 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b59ad-645d-4a46-a270-8819db760b7c None admin] RBAC: Authorizing `identity:get_user(user_id=2301e4f1cc2d4728b95a1e44f766583f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:49.830973 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1d011-7def-4025-b754-6574fc6bef1f None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:49.831879 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b59ad-645d-4a46-a270-8819db760b7c None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:49.834813 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 164/332] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:49 2022] GET /identity/v3/users/2301e4f1cc2d4728b95a1e44f766583f => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:49.840056 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b59ad-645d-4a46-a270-8819db760b7c None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:49.846256 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 169/333] 10.222.0.36 () {56 vars in 1308 bytes} [Fri Sep 2 08:55:49 2022] GET /identity/v3/role_assignments?user.id=b2d95f11f2ce4e3787be502d551497d4&scope.system=all&role.id=08c2a54a85ef4bb9892a5769ef8a5d95 => generated 510 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:49.961035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b59ad-645d-4a46-a270-8819db760b7c None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:49.999036 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939c0d9-ed15-4f80-b0a6-b0660c8663a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:50.000074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939c0d9-ed15-4f80-b0a6-b0660c8663a0 None admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:50.000613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939c0d9-ed15-4f80-b0a6-b0660c8663a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:50.001077 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939c0d9-ed15-4f80-b0a6-b0660c8663a0 None admin] PATH_INFO: `/v3/system/users/2301e4f1cc2d4728b95a1e44f766583f/roles/b1bd3fa77e3d4c83aa57374cb35fd21f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:50.002869 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939c0d9-ed15-4f80-b0a6-b0660c8663a0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2301e4f1cc2d4728b95a1e44f766583f, role_id=b1bd3fa77e3d4c83aa57374cb35fd21f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:50.004536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939c0d9-ed15-4f80-b0a6-b0660c8663a0 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:50.006385 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:50.014207 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 165/334] 10.222.0.36 () {58 vars in 1334 bytes} [Fri Sep 2 08:55:49 2022] PUT /identity/v3/system/users/2301e4f1cc2d4728b95a1e44f766583f/roles/b1bd3fa77e3d4c83aa57374cb35fd21f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 08:55:51.361673 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7015562-8659-46fe-aa75-e22cf5659d54 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:51.362111 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7015562-8659-46fe-aa75-e22cf5659d54 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:51.362286 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7015562-8659-46fe-aa75-e22cf5659d54 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:51.363842 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 170/335] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:51.369779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26630e39-7116-4b0b-82dc-c4d1180261dc None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:51.370221 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26630e39-7116-4b0b-82dc-c4d1180261dc None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:51.370393 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26630e39-7116-4b0b-82dc-c4d1180261dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:51.390019 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-26630e39-7116-4b0b-82dc-c4d1180261dc None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:51.391227 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-26630e39-7116-4b0b-82dc-c4d1180261dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:51.467463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 166/336] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:55:51.876278 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7015562-8659-46fe-aa75-e22cf5659d54 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:51.925621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd1829-b516-4a93-8a20-09ea91454c5a None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:51.925937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd1829-b516-4a93-8a20-09ea91454c5a None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:51.926172 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fd1829-b516-4a93-8a20-09ea91454c5a None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:51.928412 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 167/337] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:51.934030 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53e2f7-88f5-47c6-86cc-3b0530264c40 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:51.934310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53e2f7-88f5-47c6-86cc-3b0530264c40 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:51.934577 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53e2f7-88f5-47c6-86cc-3b0530264c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:51.968079 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-ef53e2f7-88f5-47c6-86cc-3b0530264c40 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:51.969270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ef53e2f7-88f5-47c6-86cc-3b0530264c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:51.992572 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:52.032665 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7015562-8659-46fe-aa75-e22cf5659d54 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:52.109191 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2dcead-40dd-46e1-8d1b-2f1196fe1782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:52.109840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2dcead-40dd-46e1-8d1b-2f1196fe1782 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:52.110122 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2dcead-40dd-46e1-8d1b-2f1196fe1782 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:52.110395 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2dcead-40dd-46e1-8d1b-2f1196fe1782 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:52.113296 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 168/338] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:55:52.134164 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2dcead-40dd-46e1-8d1b-2f1196fe1782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:52.139487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-ed2dcead-40dd-46e1-8d1b-2f1196fe1782 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:52.139487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:52.139487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:52.139487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:52.139487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:52.139487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:52.139487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:52.139487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:52.139487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:52.139487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:52.139487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:52.139487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:52.139487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:52.139487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:52.139487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:52.139487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:52.140375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:52.140375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:52.140375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:52.140375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:52.140375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:52.140375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:52.140375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:52.140375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:52.140375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:52.140375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:52.140375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:52.140375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:52.140375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:52.140375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:52.140375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:52.140375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:52.141168 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:52.141168 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:52.141168 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:52.141168 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:52.141168 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:52.141168 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:52.141168 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:52.141168 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:52.141168 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:52.141168 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:52.141168 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:52.141168 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:52.141168 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 171/339] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:52.147232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53e2f7-88f5-47c6-86cc-3b0530264c40 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:52.217825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84dde4-f732-4204-89c8-c8e81e7db25f None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:52.218150 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84dde4-f732-4204-89c8-c8e81e7db25f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:52.218374 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84dde4-f732-4204-89c8-c8e81e7db25f None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:52.220038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 172/340] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:52.226244 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053e101-1e75-45ba-88d8-04028c2bab20 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:52.226529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053e101-1e75-45ba-88d8-04028c2bab20 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:52.226806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053e101-1e75-45ba-88d8-04028c2bab20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:52.253507 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:52.260191 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-5053e101-1e75-45ba-88d8-04028c2bab20 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:52.261778 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5053e101-1e75-45ba-88d8-04028c2bab20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:52.325866 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53e2f7-88f5-47c6-86cc-3b0530264c40 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:52.386384 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 173/341] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:55:52.388851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b61742-dc70-447b-b898-889540756575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:52.389467 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b61742-dc70-447b-b898-889540756575 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:52.389742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b61742-dc70-447b-b898-889540756575 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:52.390011 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b61742-dc70-447b-b898-889540756575 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:52.390684 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b61742-dc70-447b-b898-889540756575 None admin] RBAC: Adding query filter params (name=member) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:52.390982 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b61742-dc70-447b-b898-889540756575 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:52.392379 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b61742-dc70-447b-b898-889540756575 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:52.401760 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 169/342] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:52.407491 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053e101-1e75-45ba-88d8-04028c2bab20 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:52.416958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fa0ee-8fbc-4663-ad5f-cd1015c73a6b None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:52.417452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fa0ee-8fbc-4663-ad5f-cd1015c73a6b None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:52.417897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fa0ee-8fbc-4663-ad5f-cd1015c73a6b None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:52.419672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 170/343] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:55:52.424678 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6a665-31c9-4acf-8b3a-0e88c5f27b92 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:52.425133 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6a665-31c9-4acf-8b3a-0e88c5f27b92 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:52.425553 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6a665-31c9-4acf-8b3a-0e88c5f27b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:52.451763 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-10b6a665-31c9-4acf-8b3a-0e88c5f27b92 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:52.452981 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-10b6a665-31c9-4acf-8b3a-0e88c5f27b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:52.544980 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:52.582911 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 171/344] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:55:52.599334 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053e101-1e75-45ba-88d8-04028c2bab20 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:52.635341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcbff1c-85fa-471f-bc3d-cbe564f9a557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:52.635949 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbff1c-85fa-471f-bc3d-cbe564f9a557 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:52.636284 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbff1c-85fa-471f-bc3d-cbe564f9a557 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:52.636618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbff1c-85fa-471f-bc3d-cbe564f9a557 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:52.647291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6a665-31c9-4acf-8b3a-0e88c5f27b92 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:52.731022 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcbff1c-85fa-471f-bc3d-cbe564f9a557 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:52.732697 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcbff1c-85fa-471f-bc3d-cbe564f9a557 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:52.771956 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:52.813534 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:52.851614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6a665-31c9-4acf-8b3a-0e88c5f27b92 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:52.854764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 174/345] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 02 08:55:52.864003 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcbff1c-85fa-471f-bc3d-cbe564f9a557 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:52.922648 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75146749-377d-4eb3-b615-4b2dd029d9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:52.923458 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75146749-377d-4eb3-b615-4b2dd029d9aa None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:52.923746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75146749-377d-4eb3-b615-4b2dd029d9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:52.924015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75146749-377d-4eb3-b615-4b2dd029d9aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:52.946736 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75146749-377d-4eb3-b615-4b2dd029d9aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:52.952102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-75146749-377d-4eb3-b615-4b2dd029d9aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:55:52.952102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:52.952102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:52.952102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:52.952102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:52.952102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:52.952102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:52.952102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:52.952102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:52.952102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:52.952102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:52.952102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:52.952102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:52.952102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:52.952102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:52.952102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:52.952959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:52.952959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:52.952959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:52.952959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:52.952959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:52.952959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:52.952959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:52.952959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:52.952959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:52.952959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:52.952959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:52.952959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:52.952959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:52.952959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:52.952959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:52.952959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:52.954138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:52.954138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:52.954138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:52.954138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:52.954138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:52.954138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:52.954138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:52.954138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:52.954138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:52.954138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:52.954138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:55:52.954138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:52.954138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 172/346] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:55:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:52.954978 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75146749-377d-4eb3-b615-4b2dd029d9aa None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:53.032742 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcbff1c-85fa-471f-bc3d-cbe564f9a557 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:53.079886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69e15c-d1d7-447d-af44-a292f06d4415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:53.081055 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69e15c-d1d7-447d-af44-a292f06d4415 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:53.081472 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69e15c-d1d7-447d-af44-a292f06d4415 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:53.081752 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69e15c-d1d7-447d-af44-a292f06d4415 None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:53.083639 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69e15c-d1d7-447d-af44-a292f06d4415 None admin] RBAC: Authorizing `identity:get_project(project_id=2d3731a0d0214a2ab2046a5d36d29aa4)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:53.085379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69e15c-d1d7-447d-af44-a292f06d4415 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:53.086236 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75146749-377d-4eb3-b615-4b2dd029d9aa None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:53.087337 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 175/347] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:52 2022] GET /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:53.088122 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69e15c-d1d7-447d-af44-a292f06d4415 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:53.108504 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:53.139062 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5621fdb-75b2-43a4-bc89-93a55cad6e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:53.139836 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5621fdb-75b2-43a4-bc89-93a55cad6e9b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:53.140180 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5621fdb-75b2-43a4-bc89-93a55cad6e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:53.140605 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5621fdb-75b2-43a4-bc89-93a55cad6e9b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:53.149623 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:53.154551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5621fdb-75b2-43a4-bc89-93a55cad6e9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:53.159762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-a5621fdb-75b2-43a4-bc89-93a55cad6e9b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 08:55:53.159762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:53.159762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:53.159762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:53.159762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:53.159762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:53.159762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:53.159762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:53.159762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:53.159762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:53.159762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:53.159762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:53.159762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:53.159762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:53.159762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:53.159762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:53.160916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:53.160916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:53.160916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:53.160916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:53.160916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:53.160916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:53.160916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:53.160916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:53.160916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:53.160916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:53.160916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:53.160916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:53.160916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:53.160916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:53.160916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:53.160916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:53.161907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:53.161907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:53.161907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:53.161907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:53.161907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:53.161907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:53.161907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:53.161907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:53.161907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:53.161907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:53.161907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 08:55:53.161907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:53.161907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 173/348] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:53.162707 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5621fdb-75b2-43a4-bc89-93a55cad6e9b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:53.230605 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69e15c-d1d7-447d-af44-a292f06d4415 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:53.267420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bce766-3101-4cd0-9d19-c7a0608de92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:53.268072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bce766-3101-4cd0-9d19-c7a0608de92b None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:53.268396 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bce766-3101-4cd0-9d19-c7a0608de92b None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:53.268674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bce766-3101-4cd0-9d19-c7a0608de92b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:53.279490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bce766-3101-4cd0-9d19-c7a0608de92b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:53.284520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-d8bce766-3101-4cd0-9d19-c7a0608de92b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:53.284520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:53.284520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:53.284520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:53.284520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:53.284520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:53.284520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:53.284520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:53.284520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:53.284520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:53.284520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:53.284520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:53.284520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:53.284520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:53.284520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:53.284520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:53.285527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:53.285527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:53.285527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:53.285527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:53.285527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:53.285527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:53.285527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:53.285527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:53.285527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:53.285527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:53.285527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:53.285527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:53.285527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:53.285527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:53.285527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:53.285527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:53.286407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:53.286407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:53.286407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:53.286407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:53.286407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:53.286407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:53.286407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:53.286407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:53.286407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:53.286407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:53.286407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:53.286407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:53.286407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 176/349] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:53.286407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bce766-3101-4cd0-9d19-c7a0608de92b None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:53.288924 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5621fdb-75b2-43a4-bc89-93a55cad6e9b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:53.326147 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13136118-f518-405b-9ee6-116697f0ef70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:53.327032 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13136118-f518-405b-9ee6-116697f0ef70 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:53.327489 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13136118-f518-405b-9ee6-116697f0ef70 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:53.327900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13136118-f518-405b-9ee6-116697f0ef70 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:53.328701 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13136118-f518-405b-9ee6-116697f0ef70 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:53.329156 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13136118-f518-405b-9ee6-116697f0ef70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:53.330662 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13136118-f518-405b-9ee6-116697f0ef70 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:53.339958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 174/350] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:55:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:53.340958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13136118-f518-405b-9ee6-116697f0ef70 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:53.376796 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:53.399365 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:53.402264 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bce766-3101-4cd0-9d19-c7a0608de92b None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:53.443479 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb10a3-dce3-4609-ad31-e4781857f80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:53.444444 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb10a3-dce3-4609-ad31-e4781857f80e None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:53.445443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb10a3-dce3-4609-ad31-e4781857f80e None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:53.445903 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fb10a3-dce3-4609-ad31-e4781857f80e None admin] PATH_INFO: `/v3/groups/5306a92295e04bd4a46d7fdfce71db73` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:53.452866 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fb10a3-dce3-4609-ad31-e4781857f80e None admin] RBAC: Authorizing `identity:get_group(group_id=5306a92295e04bd4a46d7fdfce71db73)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:53.454587 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fb10a3-dce3-4609-ad31-e4781857f80e None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:53.456775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 177/351] 10.222.0.36 () {56 vars in 1180 bytes} [Fri Sep 2 08:55:53 2022] GET /identity/v3/groups/5306a92295e04bd4a46d7fdfce71db73 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:53.457753 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb10a3-dce3-4609-ad31-e4781857f80e None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:53.459318 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13136118-f518-405b-9ee6-116697f0ef70 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:53.500443 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b58b39-cce1-49d2-ad2b-13ae4755d972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:53.501608 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b58b39-cce1-49d2-ad2b-13ae4755d972 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:53.502102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b58b39-cce1-49d2-ad2b-13ae4755d972 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:53.502574 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b58b39-cce1-49d2-ad2b-13ae4755d972 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:53.504549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b58b39-cce1-49d2-ad2b-13ae4755d972 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:53.505061 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b58b39-cce1-49d2-ad2b-13ae4755d972 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:53.507186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b58b39-cce1-49d2-ad2b-13ae4755d972 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:53.518897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 175/352] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:53.520274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b58b39-cce1-49d2-ad2b-13ae4755d972 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:53.587178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fb10a3-dce3-4609-ad31-e4781857f80e None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:53.632044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59736aa7-5269-431b-9fe7-46bb4cf89b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:53.632797 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59736aa7-5269-431b-9fe7-46bb4cf89b90 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:53.633190 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59736aa7-5269-431b-9fe7-46bb4cf89b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:53.633555 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59736aa7-5269-431b-9fe7-46bb4cf89b90 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:53.634887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59736aa7-5269-431b-9fe7-46bb4cf89b90 None admin] RBAC: Adding query filter params (name=member) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:53.635426 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59736aa7-5269-431b-9fe7-46bb4cf89b90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:53.637312 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59736aa7-5269-431b-9fe7-46bb4cf89b90 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:53.641755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:53.651433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 178/353] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:53.651716 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b58b39-cce1-49d2-ad2b-13ae4755d972 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:53.652882 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59736aa7-5269-431b-9fe7-46bb4cf89b90 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:53.690639 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:53.696204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7796904-836b-4f5c-95eb-5a257bec6d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:53.697586 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7796904-836b-4f5c-95eb-5a257bec6d55 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:53.698259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7796904-836b-4f5c-95eb-5a257bec6d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:53.698937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7796904-836b-4f5c-95eb-5a257bec6d55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:53.786750 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59736aa7-5269-431b-9fe7-46bb4cf89b90 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:53.790595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7796904-836b-4f5c-95eb-5a257bec6d55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:53.792095 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7796904-836b-4f5c-95eb-5a257bec6d55 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:53.839456 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7a5e59-ae9b-4562-a060-406708613df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:53.840192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a5e59-ae9b-4562-a060-406708613df5 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:53.840544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a5e59-ae9b-4562-a060-406708613df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:53.840887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a5e59-ae9b-4562-a060-406708613df5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:53.841623 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7a5e59-ae9b-4562-a060-406708613df5 None admin] RBAC: Adding query filter params (group.id=5306a92295e04bd4a46d7fdfce71db73, role.id=08c2a54a85ef4bb9892a5769ef8a5d95, scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:53.841933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7a5e59-ae9b-4562-a060-406708613df5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:53.843937 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7a5e59-ae9b-4562-a060-406708613df5 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:53.850952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 179/354] 10.222.0.36 () {56 vars in 1376 bytes} [Fri Sep 2 08:55:53 2022] GET /identity/v3/role_assignments?group.id=5306a92295e04bd4a46d7fdfce71db73&scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4&role.id=08c2a54a85ef4bb9892a5769ef8a5d95 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:53.852022 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7a5e59-ae9b-4562-a060-406708613df5 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:53.926292 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:53.940014 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 176/355] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:53.940889 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7796904-836b-4f5c-95eb-5a257bec6d55 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:53.991443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:53.996204 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7a5e59-ae9b-4562-a060-406708613df5 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:54.038594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79421da0-9fce-4154-b61e-4d4fa95ba5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:54.039189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79421da0-9fce-4154-b61e-4d4fa95ba5de None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:54.039470 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79421da0-9fce-4154-b61e-4d4fa95ba5de None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:54.039744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79421da0-9fce-4154-b61e-4d4fa95ba5de None admin] PATH_INFO: `/v3/users/2301e4f1cc2d4728b95a1e44f766583f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:54.040888 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79421da0-9fce-4154-b61e-4d4fa95ba5de None admin] RBAC: Authorizing `identity:get_user(user_id=2301e4f1cc2d4728b95a1e44f766583f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:54.042339 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79421da0-9fce-4154-b61e-4d4fa95ba5de None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:54.044462 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 180/356] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:55:53 2022] GET /identity/v3/users/2301e4f1cc2d4728b95a1e44f766583f => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:54.045948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb86cf-e326-4163-8769-a2258804986e None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:54.046248 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb86cf-e326-4163-8769-a2258804986e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:54.046498 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb86cf-e326-4163-8769-a2258804986e None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:54.047996 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 181/357] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:54.048704 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb86cf-e326-4163-8769-a2258804986e None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:54.077829 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7796904-836b-4f5c-95eb-5a257bec6d55 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:54.123389 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0009220-8d81-4197-8ea5-276a3499b4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:54.124008 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0009220-8d81-4197-8ea5-276a3499b4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:54.124287 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0009220-8d81-4197-8ea5-276a3499b4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:54.124564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0009220-8d81-4197-8ea5-276a3499b4b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:54.178231 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb86cf-e326-4163-8769-a2258804986e None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:54.206513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:54.218612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0009220-8d81-4197-8ea5-276a3499b4b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:54.220470 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0009220-8d81-4197-8ea5-276a3499b4b4 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:54.228152 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b4b665-acbd-4c0d-a01d-f9efd26971ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:54.228152 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4b665-acbd-4c0d-a01d-f9efd26971ff None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:54.228396 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4b665-acbd-4c0d-a01d-f9efd26971ff None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:54.228784 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b4b665-acbd-4c0d-a01d-f9efd26971ff None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:54.229956 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b4b665-acbd-4c0d-a01d-f9efd26971ff None admin] RBAC: Authorizing `identity:get_project(project_id=2d3731a0d0214a2ab2046a5d36d29aa4)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:54.231440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b4b665-acbd-4c0d-a01d-f9efd26971ff None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:54.233382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 182/358] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:54 2022] GET /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:54.234361 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b4b665-acbd-4c0d-a01d-f9efd26971ff None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:54.274647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:54.366344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 177/359] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:54.366504 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b4b665-acbd-4c0d-a01d-f9efd26971ff None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:54.366600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db89ec4-db12-42be-a305-f46c2391580b None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:54.366600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db89ec4-db12-42be-a305-f46c2391580b None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:54.366600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db89ec4-db12-42be-a305-f46c2391580b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:54.387132 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-7db89ec4-db12-42be-a305-f46c2391580b None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:54.388821 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7db89ec4-db12-42be-a305-f46c2391580b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:54.404629 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e59f5-07cf-4ad2-96c7-b6f57c30f84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:54.405255 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e59f5-07cf-4ad2-96c7-b6f57c30f84d None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:54.405521 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e59f5-07cf-4ad2-96c7-b6f57c30f84d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:54.405799 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e59f5-07cf-4ad2-96c7-b6f57c30f84d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:54.406692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e59f5-07cf-4ad2-96c7-b6f57c30f84d None admin] RBAC: Adding query filter params (role.id=b1bd3fa77e3d4c83aa57374cb35fd21f, user.id=2301e4f1cc2d4728b95a1e44f766583f, scope.system=all) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:54.407005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e59f5-07cf-4ad2-96c7-b6f57c30f84d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:54.409531 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e59f5-07cf-4ad2-96c7-b6f57c30f84d None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:54.424788 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 183/360] 10.222.0.36 () {56 vars in 1308 bytes} [Fri Sep 2 08:55:54 2022] GET /identity/v3/role_assignments?user.id=2301e4f1cc2d4728b95a1e44f766583f&scope.system=all&role.id=b1bd3fa77e3d4c83aa57374cb35fd21f => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:54.425602 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3a414-7c53-4866-88bf-96d815c5f85e None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:54.425951 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3a414-7c53-4866-88bf-96d815c5f85e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:54.426502 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e3a414-7c53-4866-88bf-96d815c5f85e None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:54.428596 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 184/361] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:54.429293 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3a414-7c53-4866-88bf-96d815c5f85e None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:54.487575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 178/362] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:54.489374 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db89ec4-db12-42be-a305-f46c2391580b None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:54.537286 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:54.541082 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:54.562749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e3a414-7c53-4866-88bf-96d815c5f85e None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:54.604475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582dc35-6111-477f-8c54-83f663699c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:54.605167 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582dc35-6111-477f-8c54-83f663699c00 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:54.605450 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582dc35-6111-477f-8c54-83f663699c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:54.610510 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582dc35-6111-477f-8c54-83f663699c00 None admin] PATH_INFO: `/v3/groups/5306a92295e04bd4a46d7fdfce71db73` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:54.610510 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582dc35-6111-477f-8c54-83f663699c00 None admin] RBAC: Authorizing `identity:get_group(group_id=5306a92295e04bd4a46d7fdfce71db73)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:54.610510 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582dc35-6111-477f-8c54-83f663699c00 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:54.610763 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 185/363] 10.222.0.36 () {56 vars in 1180 bytes} [Fri Sep 2 08:55:54 2022] GET /identity/v3/groups/5306a92295e04bd4a46d7fdfce71db73 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:54.611741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fea0b9-29bd-4746-a3db-62d43aa8c73b None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:54.612018 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fea0b9-29bd-4746-a3db-62d43aa8c73b None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:54.612287 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fea0b9-29bd-4746-a3db-62d43aa8c73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:54.637027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-91fea0b9-29bd-4746-a3db-62d43aa8c73b None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:54.638641 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-91fea0b9-29bd-4746-a3db-62d43aa8c73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:54.714023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 186/364] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:54.714933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fea0b9-29bd-4746-a3db-62d43aa8c73b None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:54.715208 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db89ec4-db12-42be-a305-f46c2391580b None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:54.753049 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97151b81-cbf3-443c-9853-bbfc4989480b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:54.754050 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97151b81-cbf3-443c-9853-bbfc4989480b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:54.754526 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97151b81-cbf3-443c-9853-bbfc4989480b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:54.754995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97151b81-cbf3-443c-9853-bbfc4989480b None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:54.756898 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97151b81-cbf3-443c-9853-bbfc4989480b None admin] RBAC: Authorizing `identity:get_project(project_id=78733ddb1b714faf83957113a15c51bc)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:54.758539 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97151b81-cbf3-443c-9853-bbfc4989480b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:54.760407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 179/365] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:54 2022] GET /identity/v3/projects/78733ddb1b714faf83957113a15c51bc => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:54.768492 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97151b81-cbf3-443c-9853-bbfc4989480b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:54.808571 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:54.843899 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fea0b9-29bd-4746-a3db-62d43aa8c73b None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:54.887877 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:54.894521 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb7f8a-3886-4202-a9e9-2dc4efb2da9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:54.895349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb7f8a-3886-4202-a9e9-2dc4efb2da9d None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:54.895638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb7f8a-3886-4202-a9e9-2dc4efb2da9d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:54.895961 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb7f8a-3886-4202-a9e9-2dc4efb2da9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:54.897198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb7f8a-3886-4202-a9e9-2dc4efb2da9d None admin] RBAC: Adding query filter params (group.id=5306a92295e04bd4a46d7fdfce71db73, role.id=9c6920b8e6f744a6a80371b4218605fd, scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:54.897506 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb7f8a-3886-4202-a9e9-2dc4efb2da9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:54.898682 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97151b81-cbf3-443c-9853-bbfc4989480b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:54.898971 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdb7f8a-3886-4202-a9e9-2dc4efb2da9d None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:54.904764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 187/366] 10.222.0.36 () {56 vars in 1376 bytes} [Fri Sep 2 08:55:54 2022] GET /identity/v3/role_assignments?group.id=5306a92295e04bd4a46d7fdfce71db73&scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4&role.id=9c6920b8e6f744a6a80371b4218605fd => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:54.937153 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156bb671-aee6-42c7-9df5-25faaac0854a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:54.937796 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156bb671-aee6-42c7-9df5-25faaac0854a None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:54.938072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156bb671-aee6-42c7-9df5-25faaac0854a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:54.938340 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156bb671-aee6-42c7-9df5-25faaac0854a None admin] PATH_INFO: `/v3/groups/5306a92295e04bd4a46d7fdfce71db73` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:54.939506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156bb671-aee6-42c7-9df5-25faaac0854a None admin] RBAC: Authorizing `identity:get_group(group_id=5306a92295e04bd4a46d7fdfce71db73)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:54.941034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156bb671-aee6-42c7-9df5-25faaac0854a None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:54.943642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 180/367] 10.222.0.36 () {56 vars in 1180 bytes} [Fri Sep 2 08:55:54 2022] GET /identity/v3/groups/5306a92295e04bd4a46d7fdfce71db73 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:54.949545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb7f8a-3886-4202-a9e9-2dc4efb2da9d None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:55.053929 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156bb671-aee6-42c7-9df5-25faaac0854a None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:55.102429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb7f8a-3886-4202-a9e9-2dc4efb2da9d None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:55.163520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f2468-0e06-4263-b74e-d8e014121788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:55.164045 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f2468-0e06-4263-b74e-d8e014121788 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:55.164320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f2468-0e06-4263-b74e-d8e014121788 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:55.164587 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f2468-0e06-4263-b74e-d8e014121788 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:55.165745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f2468-0e06-4263-b74e-d8e014121788 None admin] RBAC: Adding query filter params (group.id=5306a92295e04bd4a46d7fdfce71db73, role.id=08c2a54a85ef4bb9892a5769ef8a5d95, scope.project.id=78733ddb1b714faf83957113a15c51bc) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:55.166069 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f2468-0e06-4263-b74e-d8e014121788 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:55.167560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f2468-0e06-4263-b74e-d8e014121788 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:55.175167 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 188/368] 10.222.0.36 () {56 vars in 1376 bytes} [Fri Sep 2 08:55:54 2022] GET /identity/v3/role_assignments?group.id=5306a92295e04bd4a46d7fdfce71db73&scope.project.id=78733ddb1b714faf83957113a15c51bc&role.id=08c2a54a85ef4bb9892a5769ef8a5d95 => generated 260 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:55.176348 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f2468-0e06-4263-b74e-d8e014121788 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:55.198273 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:55.201991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156bb671-aee6-42c7-9df5-25faaac0854a None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:55.274432 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbb6d6-7869-42bf-ad9f-a47a0b3a1a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:55.275097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbb6d6-7869-42bf-ad9f-a47a0b3a1a69 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:55.275377 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbb6d6-7869-42bf-ad9f-a47a0b3a1a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:55.275647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbbb6d6-7869-42bf-ad9f-a47a0b3a1a69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:55.297140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbbb6d6-7869-42bf-ad9f-a47a0b3a1a69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:55.302647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-3cbbb6d6-7869-42bf-ad9f-a47a0b3a1a69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:55:55.302647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:55.302647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:55.302647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:55.302647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:55.302647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:55.302647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:55.302647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:55.302647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:55.302647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:55.302647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:55.302647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:55.302647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:55.302647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:55.302647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:55.302647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:55.303624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:55.303624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:55.303624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:55.303624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:55.303624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:55.303624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:55.303624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:55.303624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:55.303624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:55.303624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:55.303624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:55.303624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:55.303624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:55.303624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:55.303624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:55.303624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:55.304411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:55.304411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:55.304411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:55.304411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:55.304411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:55.304411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:55.304411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:55.304411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:55.304411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:55.304411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:55.304411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:55:55.304411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:55.310148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 181/369] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:55:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:55:55.319135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbb6d6-7869-42bf-ad9f-a47a0b3a1a69 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:55.363660 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:55.374834 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f2468-0e06-4263-b74e-d8e014121788 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:55.423319 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179ed31-ccea-4d87-b8d1-f99b14ae41ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:55.423916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179ed31-ccea-4d87-b8d1-f99b14ae41ce None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:55.424172 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179ed31-ccea-4d87-b8d1-f99b14ae41ce None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:55.424393 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179ed31-ccea-4d87-b8d1-f99b14ae41ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:55.436917 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e179ed31-ccea-4d87-b8d1-f99b14ae41ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:55.443874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-e179ed31-ccea-4d87-b8d1-f99b14ae41ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:55:55.443874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:55.443874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:55.443874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:55.443874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:55.443874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:55.443874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:55.443874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:55.443874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:55.443874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:55.443874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:55.443874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:55.443874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:55.443874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:55.443874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:55.443874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:55.444793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:55.444793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:55.444793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:55.444793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:55.444793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:55.444793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:55.444793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:55.444793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:55.444793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:55.444793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:55.444793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:55.444793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:55.444793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:55.444793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:55.444793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:55.444793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:55.445579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:55.445579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:55.445579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:55.445579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:55.445579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:55.445579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:55.445579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:55.445579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:55.445579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:55.445579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:55.445579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:55:55.445579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:55.445579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 189/370] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:55:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:55.450987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179ed31-ccea-4d87-b8d1-f99b14ae41ce None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:55.468620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbbb6d6-7869-42bf-ad9f-a47a0b3a1a69 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:55.510629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246c91f-1db1-417e-80fb-2ebce0ffcd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:55.511457 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246c91f-1db1-417e-80fb-2ebce0ffcd03 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:55.511633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246c91f-1db1-417e-80fb-2ebce0ffcd03 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:55.511815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246c91f-1db1-417e-80fb-2ebce0ffcd03 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:55.512491 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246c91f-1db1-417e-80fb-2ebce0ffcd03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:55.512793 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246c91f-1db1-417e-80fb-2ebce0ffcd03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:55.514307 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246c91f-1db1-417e-80fb-2ebce0ffcd03 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:55.523522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 182/371] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:55.539486 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:55.544155 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246c91f-1db1-417e-80fb-2ebce0ffcd03 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:55.567688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179ed31-ccea-4d87-b8d1-f99b14ae41ce None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:55.603986 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f6ef2-0335-4283-81f2-1e25f4a7cc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:55.604599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f6ef2-0335-4283-81f2-1e25f4a7cc76 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:55.604875 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f6ef2-0335-4283-81f2-1e25f4a7cc76 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:55.605257 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f6ef2-0335-4283-81f2-1e25f4a7cc76 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:55.605924 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f6ef2-0335-4283-81f2-1e25f4a7cc76 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:55.606226 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f6ef2-0335-4283-81f2-1e25f4a7cc76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:55.607741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f6ef2-0335-4283-81f2-1e25f4a7cc76 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:55.622769 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 190/372] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:55:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:55.624136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f6ef2-0335-4283-81f2-1e25f4a7cc76 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:55.656301 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:55.676982 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246c91f-1db1-417e-80fb-2ebce0ffcd03 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:55.712850 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb8423-c27f-40e1-b119-1dc2a3f40096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:55.713622 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb8423-c27f-40e1-b119-1dc2a3f40096 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:55.714002 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb8423-c27f-40e1-b119-1dc2a3f40096 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:55.714376 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb8423-c27f-40e1-b119-1dc2a3f40096 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:55.735542 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f6ef2-0335-4283-81f2-1e25f4a7cc76 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:55.773808 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7745f2-ce9b-40bd-b319-6c618ff0d1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:55.774515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7745f2-ce9b-40bd-b319-6c618ff0d1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:55.774794 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7745f2-ce9b-40bd-b319-6c618ff0d1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:55.775062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7745f2-ce9b-40bd-b319-6c618ff0d1c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:55.775944 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:55.794868 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb8423-c27f-40e1-b119-1dc2a3f40096 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:55.796426 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceb8423-c27f-40e1-b119-1dc2a3f40096 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:55.860141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7745f2-ce9b-40bd-b319-6c618ff0d1c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:55.861800 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7745f2-ce9b-40bd-b319-6c618ff0d1c4 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:55.984682 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:56.016782 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 183/373] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:56.023692 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb8423-c27f-40e1-b119-1dc2a3f40096 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:56.030587 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 191/374] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:55:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:55:56.036489 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7745f2-ce9b-40bd-b319-6c618ff0d1c4 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:56.081618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:56.168337 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb8423-c27f-40e1-b119-1dc2a3f40096 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:56.179041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7745f2-ce9b-40bd-b319-6c618ff0d1c4 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:56.218622 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c08225-59f5-4bc0-be70-9e05553303dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:56.219288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c08225-59f5-4bc0-be70-9e05553303dc None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:56.219569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c08225-59f5-4bc0-be70-9e05553303dc None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:56.219838 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c08225-59f5-4bc0-be70-9e05553303dc None admin] PATH_INFO: `/v3/projects/32923700004a4c11b8926f21bae6d460` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:56.220974 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c08225-59f5-4bc0-be70-9e05553303dc None admin] RBAC: Authorizing `identity:get_project(project_id=32923700004a4c11b8926f21bae6d460)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:56.222407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c08225-59f5-4bc0-be70-9e05553303dc None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:56.224402 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 184/375] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:56 2022] GET /identity/v3/projects/32923700004a4c11b8926f21bae6d460 => generated 344 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:56.225239 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2135a-5d37-43e9-bdd3-3350d242bcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:56.225517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a95f2a-c4ef-42f4-9aae-f62eb5d0b425 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:56.225798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a95f2a-c4ef-42f4-9aae-f62eb5d0b425 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:56.226068 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a95f2a-c4ef-42f4-9aae-f62eb5d0b425 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:56.226727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2135a-5d37-43e9-bdd3-3350d242bcb7 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:56.227245 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2135a-5d37-43e9-bdd3-3350d242bcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:56.227582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 185/376] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:56.227938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2135a-5d37-43e9-bdd3-3350d242bcb7 None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:56.229504 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d2135a-5d37-43e9-bdd3-3350d242bcb7 None admin] RBAC: Authorizing `identity:get_project(project_id=78733ddb1b714faf83957113a15c51bc)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:56.231842 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a95f2a-c4ef-42f4-9aae-f62eb5d0b425 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:56.232200 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d2135a-5d37-43e9-bdd3-3350d242bcb7 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:56.234404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 192/377] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:56 2022] GET /identity/v3/projects/78733ddb1b714faf83957113a15c51bc => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:56.236952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf05ca-4a84-46d2-8349-320fd511bf8e None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:56.237457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf05ca-4a84-46d2-8349-320fd511bf8e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:56.237960 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf05ca-4a84-46d2-8349-320fd511bf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:56.260565 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-21bf05ca-4a84-46d2-8349-320fd511bf8e None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:56.262532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-21bf05ca-4a84-46d2-8349-320fd511bf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:56.315669 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:56.344266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 193/378] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:56.345153 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf05ca-4a84-46d2-8349-320fd511bf8e None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:56.356389 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a95f2a-c4ef-42f4-9aae-f62eb5d0b425 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:56.395866 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49abea19-91ea-4477-a565-486e3a70b13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:56.397006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49abea19-91ea-4477-a565-486e3a70b13b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:56.397591 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:56.397905 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49abea19-91ea-4477-a565-486e3a70b13b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:56.398199 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49abea19-91ea-4477-a565-486e3a70b13b None admin] PATH_INFO: `/v3/groups/9f4e02017dac436eae4c8264d70ddfb1` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:56.406860 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49abea19-91ea-4477-a565-486e3a70b13b None admin] RBAC: Authorizing `identity:get_group(group_id=9f4e02017dac436eae4c8264d70ddfb1)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:56.408525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49abea19-91ea-4477-a565-486e3a70b13b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:56.410694 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 186/379] 10.222.0.36 () {56 vars in 1180 bytes} [Fri Sep 2 08:55:56 2022] GET /identity/v3/groups/9f4e02017dac436eae4c8264d70ddfb1 => generated 229 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:56.420071 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49abea19-91ea-4477-a565-486e3a70b13b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:56.514539 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf05ca-4a84-46d2-8349-320fd511bf8e None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:56.579958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a4a86-b39c-4420-8391-c76fc09a5a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:56.580594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a4a86-b39c-4420-8391-c76fc09a5a35 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:56.580866 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a4a86-b39c-4420-8391-c76fc09a5a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:56.581131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a4a86-b39c-4420-8391-c76fc09a5a35 None admin] PATH_INFO: `/v3/groups/5306a92295e04bd4a46d7fdfce71db73` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:56.582260 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a4a86-b39c-4420-8391-c76fc09a5a35 None admin] RBAC: Authorizing `identity:get_group(group_id=5306a92295e04bd4a46d7fdfce71db73)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:56.583766 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a4a86-b39c-4420-8391-c76fc09a5a35 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:56.585325 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 194/380] 10.222.0.36 () {56 vars in 1180 bytes} [Fri Sep 2 08:55:56 2022] GET /identity/v3/groups/5306a92295e04bd4a46d7fdfce71db73 => generated 226 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:56.593847 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a4a86-b39c-4420-8391-c76fc09a5a35 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:56.610407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49abea19-91ea-4477-a565-486e3a70b13b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:56.633011 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:56.671225 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4fe25-ac79-4b53-b816-c88abe8aaf16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:56.671841 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4fe25-ac79-4b53-b816-c88abe8aaf16 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:56.672122 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4fe25-ac79-4b53-b816-c88abe8aaf16 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:56.672393 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4fe25-ac79-4b53-b816-c88abe8aaf16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:56.673128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4fe25-ac79-4b53-b816-c88abe8aaf16 None admin] RBAC: Adding query filter params (group.id=9f4e02017dac436eae4c8264d70ddfb1, role.id=a0a76de72fba4036bc1e3e64df717c7f, scope.project.id=32923700004a4c11b8926f21bae6d460) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:56.673434 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4fe25-ac79-4b53-b816-c88abe8aaf16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:56.674870 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4fe25-ac79-4b53-b816-c88abe8aaf16 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:56.683982 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 187/381] 10.222.0.36 () {56 vars in 1376 bytes} [Fri Sep 2 08:55:56 2022] GET /identity/v3/role_assignments?group.id=9f4e02017dac436eae4c8264d70ddfb1&scope.project.id=32923700004a4c11b8926f21bae6d460&role.id=a0a76de72fba4036bc1e3e64df717c7f => generated 260 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:56.703319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4fe25-ac79-4b53-b816-c88abe8aaf16 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:56.777603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:56.790004 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a4a86-b39c-4420-8391-c76fc09a5a35 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:56.855544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f82e6-0aee-4271-8917-619d5e4dffb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:56.856562 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f82e6-0aee-4271-8917-619d5e4dffb2 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:56.857018 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f82e6-0aee-4271-8917-619d5e4dffb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:56.857437 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f82e6-0aee-4271-8917-619d5e4dffb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:56.858329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f82e6-0aee-4271-8917-619d5e4dffb2 None admin] RBAC: Adding query filter params (group.id=5306a92295e04bd4a46d7fdfce71db73, role.id=9c6920b8e6f744a6a80371b4218605fd, scope.project.id=78733ddb1b714faf83957113a15c51bc) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:56.858789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f82e6-0aee-4271-8917-619d5e4dffb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:56.861214 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f82e6-0aee-4271-8917-619d5e4dffb2 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:56.874530 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 195/382] 10.222.0.36 () {56 vars in 1376 bytes} [Fri Sep 2 08:55:56 2022] GET /identity/v3/role_assignments?group.id=5306a92295e04bd4a46d7fdfce71db73&scope.project.id=78733ddb1b714faf83957113a15c51bc&role.id=9c6920b8e6f744a6a80371b4218605fd => generated 260 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:56.906241 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4fe25-ac79-4b53-b816-c88abe8aaf16 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:56.945582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae242e-d524-4a3b-8e99-0b8e543e307a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:56.946250 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae242e-d524-4a3b-8e99-0b8e543e307a None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:56.946532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae242e-d524-4a3b-8e99-0b8e543e307a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:56.946803 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae242e-d524-4a3b-8e99-0b8e543e307a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:56.959040 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae242e-d524-4a3b-8e99-0b8e543e307a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:56.964745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-10ae242e-d524-4a3b-8e99-0b8e543e307a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:56.964745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:56.964745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:56.964745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:56.964745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:56.964745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:56.964745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:56.964745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:56.964745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:56.964745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:56.964745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:56.964745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:56.964745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:56.964745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:56.964745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:56.964745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:56.965710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:56.965710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:56.965710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:56.965710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:56.965710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:56.965710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:56.965710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:56.965710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:56.965710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:56.965710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:56.965710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:56.965710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:56.965710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:56.965710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:56.965710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:56.965710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:56.966499 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:56.966499 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:56.966499 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:56.966499 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:56.966499 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:56.966499 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:56.966499 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:56.966499 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:56.966499 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:56.966499 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:56.966499 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:56.966499 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:55:56.966499 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 188/383] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:56.976218 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f82e6-0aee-4271-8917-619d5e4dffb2 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:57.094658 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f82e6-0aee-4271-8917-619d5e4dffb2 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:57.135659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90236ca1-55fe-492e-aebb-f7586eaf877f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:57.136263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90236ca1-55fe-492e-aebb-f7586eaf877f None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:57.136539 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90236ca1-55fe-492e-aebb-f7586eaf877f None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:57.136804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90236ca1-55fe-492e-aebb-f7586eaf877f None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:57.137468 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90236ca1-55fe-492e-aebb-f7586eaf877f None admin] RBAC: Adding query filter params (name=member) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:57.137764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90236ca1-55fe-492e-aebb-f7586eaf877f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:57.142508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90236ca1-55fe-492e-aebb-f7586eaf877f None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:57.151772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 196/384] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:57.157411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae242e-d524-4a3b-8e99-0b8e543e307a None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:57.218271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:57.290450 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae242e-d524-4a3b-8e99-0b8e543e307a None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:57.335050 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2785f-55d4-44eb-ac13-34657aeb21b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:57.335715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2785f-55d4-44eb-ac13-34657aeb21b9 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:57.335997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2785f-55d4-44eb-ac13-34657aeb21b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:57.336275 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2785f-55d4-44eb-ac13-34657aeb21b9 None admin] PATH_INFO: `/v3/groups/5306a92295e04bd4a46d7fdfce71db73` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:57.339751 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2785f-55d4-44eb-ac13-34657aeb21b9 None admin] RBAC: Authorizing `identity:get_group(group_id=5306a92295e04bd4a46d7fdfce71db73)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:57.341214 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2785f-55d4-44eb-ac13-34657aeb21b9 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:57.342832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 189/385] 10.222.0.36 () {56 vars in 1180 bytes} [Fri Sep 2 08:55:57 2022] GET /identity/v3/groups/5306a92295e04bd4a46d7fdfce71db73 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:57.348534 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90236ca1-55fe-492e-aebb-f7586eaf877f None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:57.383597 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:57.473370 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90236ca1-55fe-492e-aebb-f7586eaf877f None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:57.517619 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67b9e5-a2ec-401f-a1f2-e810eee85bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:57.518263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67b9e5-a2ec-401f-a1f2-e810eee85bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:57.518506 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67b9e5-a2ec-401f-a1f2-e810eee85bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:57.518783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67b9e5-a2ec-401f-a1f2-e810eee85bf7 None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:57.519927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e67b9e5-a2ec-401f-a1f2-e810eee85bf7 None admin] RBAC: Authorizing `identity:get_project(project_id=2d3731a0d0214a2ab2046a5d36d29aa4)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:57.521903 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e67b9e5-a2ec-401f-a1f2-e810eee85bf7 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:57.523570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 197/386] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:57 2022] GET /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:57.529366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2785f-55d4-44eb-ac13-34657aeb21b9 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:57.659644 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488eb3b-2791-4967-a178-2ba255f311f2 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:57.659990 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488eb3b-2791-4967-a178-2ba255f311f2 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:57.660185 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488eb3b-2791-4967-a178-2ba255f311f2 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:57.662350 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 198/387] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:57.663950 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2785f-55d4-44eb-ac13-34657aeb21b9 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:57.667400 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d256a7-f458-4d4b-8d4d-fb2e4d0c9864 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:57.667687 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d256a7-f458-4d4b-8d4d-fb2e4d0c9864 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:57.667961 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d256a7-f458-4d4b-8d4d-fb2e4d0c9864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:57.675606 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:57.689163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-21d256a7-f458-4d4b-8d4d-fb2e4d0c9864 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:57.690359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-21d256a7-f458-4d4b-8d4d-fb2e4d0c9864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:57.714874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ee909-6e10-48be-aa05-dde82b0102d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:57.715753 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ee909-6e10-48be-aa05-dde82b0102d6 None admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:57.716040 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ee909-6e10-48be-aa05-dde82b0102d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:57.716314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ee909-6e10-48be-aa05-dde82b0102d6 None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4/groups/5306a92295e04bd4a46d7fdfce71db73/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:57.718535 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ee909-6e10-48be-aa05-dde82b0102d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d3731a0d0214a2ab2046a5d36d29aa4, group_id=5306a92295e04bd4a46d7fdfce71db73, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:57.720510 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ee909-6e10-48be-aa05-dde82b0102d6 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:57.742019 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 190/388] 10.222.0.36 () {58 vars in 1442 bytes} [Fri Sep 2 08:55:57 2022] PUT /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4/groups/5306a92295e04bd4a46d7fdfce71db73/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:55:57.744688 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03eb713-16f3-45d4-ac93-7eda0b9425df None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:57.744974 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03eb713-16f3-45d4-ac93-7eda0b9425df None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:57.745900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03eb713-16f3-45d4-ac93-7eda0b9425df None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:57.747811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 191/389] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:57.755240 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b84309-2bea-4da8-8237-f8dce4293833 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:57.755673 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b84309-2bea-4da8-8237-f8dce4293833 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:57.756078 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b84309-2bea-4da8-8237-f8dce4293833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:57.781067 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-b0b84309-2bea-4da8-8237-f8dce4293833 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:57.782468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b0b84309-2bea-4da8-8237-f8dce4293833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:57.785048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 199/390] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:55:57.886241 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 192/391] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:58.130862 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d256a7-f458-4d4b-8d4d-fb2e4d0c9864 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:58.156067 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:58.242150 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d256a7-f458-4d4b-8d4d-fb2e4d0c9864 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:58.287374 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db801d84-9836-443b-a452-60282d404f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:58.288017 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db801d84-9836-443b-a452-60282d404f08 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:58.288293 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db801d84-9836-443b-a452-60282d404f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:58.288558 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db801d84-9836-443b-a452-60282d404f08 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:58.301801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db801d84-9836-443b-a452-60282d404f08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:58.306681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-db801d84-9836-443b-a452-60282d404f08 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:58.306681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:58.306681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:58.306681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:58.306681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:58.306681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:58.306681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:58.306681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:58.306681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:58.306681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:58.306681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:58.306681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:58.306681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:58.306681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:58.306681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:58.306681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:58.307589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:58.307589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:58.307589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:58.307589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:58.307589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:58.307589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:58.307589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:58.307589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:58.307589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:58.307589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:58.307589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:58.307589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:58.307589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:58.307589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:58.307589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:58.307589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:58.308757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:58.308757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:58.308757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:58.308757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:58.308757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:58.308757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:58.308757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:58.308757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:58.308757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:58.308757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:58.308757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:55:58.308757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:58.308757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 200/392] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:55:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:55:58.314201 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b84309-2bea-4da8-8237-f8dce4293833 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:58.348286 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db801d84-9836-443b-a452-60282d404f08 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:58.499091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:58.507482 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:58.510107 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b84309-2bea-4da8-8237-f8dce4293833 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:58.544844 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db801d84-9836-443b-a452-60282d404f08 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:58.575285 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04404d-7328-47f2-aa2d-84af0aaed5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:58.575937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04404d-7328-47f2-aa2d-84af0aaed5d1 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:58.576227 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04404d-7328-47f2-aa2d-84af0aaed5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:58.576494 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04404d-7328-47f2-aa2d-84af0aaed5d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:58.577679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04404d-7328-47f2-aa2d-84af0aaed5d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:58.577997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04404d-7328-47f2-aa2d-84af0aaed5d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:58.579400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04404d-7328-47f2-aa2d-84af0aaed5d1 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:58.592151 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 193/393] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:55:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:58.598601 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04404d-7328-47f2-aa2d-84af0aaed5d1 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:58.617160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c5c7e-67ac-4038-a2e1-b8093c5011f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:58.617808 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c5c7e-67ac-4038-a2e1-b8093c5011f0 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:58.618083 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c5c7e-67ac-4038-a2e1-b8093c5011f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:58.618352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c5c7e-67ac-4038-a2e1-b8093c5011f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:58.640037 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c5c7e-67ac-4038-a2e1-b8093c5011f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:55:58.645368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-d37c5c7e-67ac-4038-a2e1-b8093c5011f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:55:58.645368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:55:58.645368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:55:58.645368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:55:58.645368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:55:58.645368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:55:58.645368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:55:58.645368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:55:58.645368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:55:58.645368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:55:58.645368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:55:58.645368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:55:58.645368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:55:58.645368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:55:58.645368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:55:58.645368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:55:58.646938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:55:58.646938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:55:58.646938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:55:58.646938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:55:58.646938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:55:58.646938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:55:58.646938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:55:58.646938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:55:58.646938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:55:58.646938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:55:58.646938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:55:58.646938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:55:58.646938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:55:58.646938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:55:58.646938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:55:58.646938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:55:58.647781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:55:58.647781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:55:58.647781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:55:58.647781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:55:58.647781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:55:58.647781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:55:58.647781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:55:58.647781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:55:58.647781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:55:58.647781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:55:58.647781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:55:58.647781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:55:58.652278 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 201/394] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:55:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:55:58.653554 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c5c7e-67ac-4038-a2e1-b8093c5011f0 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:58.772169 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04404d-7328-47f2-aa2d-84af0aaed5d1 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:58.785348 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c5c7e-67ac-4038-a2e1-b8093c5011f0 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:58.823523 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb5509-63c3-493d-84c0-c86a02a9fc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:58.824529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb5509-63c3-493d-84c0-c86a02a9fc4a None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:58.825266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb5509-63c3-493d-84c0-c86a02a9fc4a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:58.825688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb5509-63c3-493d-84c0-c86a02a9fc4a None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:58.826658 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb5509-63c3-493d-84c0-c86a02a9fc4a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:55:58.827336 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb5509-63c3-493d-84c0-c86a02a9fc4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:58.828854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb5509-63c3-493d-84c0-c86a02a9fc4a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:58.833114 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd155179-465e-4165-a7e7-b4b965cf50b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:58.834413 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd155179-465e-4165-a7e7-b4b965cf50b5 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:58.834781 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd155179-465e-4165-a7e7-b4b965cf50b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:58.835171 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd155179-465e-4165-a7e7-b4b965cf50b5 None admin] PATH_INFO: `/v3/groups/5306a92295e04bd4a46d7fdfce71db73` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:58.839271 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 202/395] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:55:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:58.839979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd155179-465e-4165-a7e7-b4b965cf50b5 None admin] RBAC: Authorizing `identity:get_group(group_id=5306a92295e04bd4a46d7fdfce71db73)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:58.841530 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd155179-465e-4165-a7e7-b4b965cf50b5 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:58.847232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 194/396] 10.222.0.36 () {56 vars in 1180 bytes} [Fri Sep 2 08:55:58 2022] GET /identity/v3/groups/5306a92295e04bd4a46d7fdfce71db73 => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:58.848423 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb5509-63c3-493d-84c0-c86a02a9fc4a None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:58.851097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd155179-465e-4165-a7e7-b4b965cf50b5 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:58.853996 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:58.920282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:58.962317 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb5509-63c3-493d-84c0-c86a02a9fc4a None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:59.007551 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab567b-28e5-4d6b-a4ac-30549cd0cab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:59.009552 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab567b-28e5-4d6b-a4ac-30549cd0cab0 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:59.009833 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab567b-28e5-4d6b-a4ac-30549cd0cab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:59.010106 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab567b-28e5-4d6b-a4ac-30549cd0cab0 None admin] PATH_INFO: `/v3/groups/5306a92295e04bd4a46d7fdfce71db73` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:59.013545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab567b-28e5-4d6b-a4ac-30549cd0cab0 None admin] RBAC: Authorizing `identity:get_group(group_id=5306a92295e04bd4a46d7fdfce71db73)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:59.017236 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab567b-28e5-4d6b-a4ac-30549cd0cab0 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:59.019051 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 203/397] 10.222.0.36 () {56 vars in 1180 bytes} [Fri Sep 2 08:55:58 2022] GET /identity/v3/groups/5306a92295e04bd4a46d7fdfce71db73 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:55:59.025493 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab567b-28e5-4d6b-a4ac-30549cd0cab0 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:59.053292 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd155179-465e-4165-a7e7-b4b965cf50b5 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:59.127778 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339bfcf-baed-4fba-a232-4b69f1d1b808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:59.128418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339bfcf-baed-4fba-a232-4b69f1d1b808 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:59.128695 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339bfcf-baed-4fba-a232-4b69f1d1b808 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:59.128968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339bfcf-baed-4fba-a232-4b69f1d1b808 None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:59.134215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339bfcf-baed-4fba-a232-4b69f1d1b808 None admin] RBAC: Authorizing `identity:get_project(project_id=78733ddb1b714faf83957113a15c51bc)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:59.135690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0339bfcf-baed-4fba-a232-4b69f1d1b808 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:59.140415 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 195/398] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:58 2022] GET /identity/v3/projects/78733ddb1b714faf83957113a15c51bc => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:59.151343 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339bfcf-baed-4fba-a232-4b69f1d1b808 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:59.175484 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:59.224966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab567b-28e5-4d6b-a4ac-30549cd0cab0 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:59.274329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15426d81-a53d-4021-be08-67f317ef96f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:59.274966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15426d81-a53d-4021-be08-67f317ef96f6 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:59.275388 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15426d81-a53d-4021-be08-67f317ef96f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:59.275722 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15426d81-a53d-4021-be08-67f317ef96f6 None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:59.276863 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15426d81-a53d-4021-be08-67f317ef96f6 None admin] RBAC: Authorizing `identity:get_project(project_id=2d3731a0d0214a2ab2046a5d36d29aa4)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:59.278316 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15426d81-a53d-4021-be08-67f317ef96f6 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:59.280344 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 204/399] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:55:59 2022] GET /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:55:59.281507 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b76a9-b676-4563-9061-bb6d22fdcc56 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:59.281783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b76a9-b676-4563-9061-bb6d22fdcc56 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:59.282052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b76a9-b676-4563-9061-bb6d22fdcc56 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:59.283531 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 205/400] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:59.286868 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b76a9-b676-4563-9061-bb6d22fdcc56 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:59.314563 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:59.324627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339bfcf-baed-4fba-a232-4b69f1d1b808 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:59.379159 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da121481-659d-4f81-b0ab-6856940528be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:59.379829 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da121481-659d-4f81-b0ab-6856940528be None admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:59.380643 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da121481-659d-4f81-b0ab-6856940528be None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:59.380919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da121481-659d-4f81-b0ab-6856940528be None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc/groups/5306a92295e04bd4a46d7fdfce71db73/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:59.382936 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da121481-659d-4f81-b0ab-6856940528be None admin] RBAC: Authorizing `identity:create_grant(project_id=78733ddb1b714faf83957113a15c51bc, group_id=5306a92295e04bd4a46d7fdfce71db73, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:59.384477 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da121481-659d-4f81-b0ab-6856940528be None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:59.404042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 196/401] 10.222.0.36 () {58 vars in 1442 bytes} [Fri Sep 2 08:55:59 2022] PUT /identity/v3/projects/78733ddb1b714faf83957113a15c51bc/groups/5306a92295e04bd4a46d7fdfce71db73/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:55:59.405339 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8a857-43d1-4b58-8d75-7d2044948de8 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:59.405623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8a857-43d1-4b58-8d75-7d2044948de8 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:59.405951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8a857-43d1-4b58-8d75-7d2044948de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:59.423322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b76a9-b676-4563-9061-bb6d22fdcc56 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:55:59.435622 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-26e8a857-43d1-4b58-8d75-7d2044948de8 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:59.436890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-26e8a857-43d1-4b58-8d75-7d2044948de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:59.467463 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e094202-a649-4c4e-8a5b-015a7ce32ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:55:59.468743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e094202-a649-4c4e-8a5b-015a7ce32ded None admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:59.469078 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e094202-a649-4c4e-8a5b-015a7ce32ded None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:59.469644 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e094202-a649-4c4e-8a5b-015a7ce32ded None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4/groups/5306a92295e04bd4a46d7fdfce71db73/roles/9c6920b8e6f744a6a80371b4218605fd` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:59.472734 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e094202-a649-4c4e-8a5b-015a7ce32ded None admin] RBAC: Authorizing `identity:create_grant(project_id=2d3731a0d0214a2ab2046a5d36d29aa4, group_id=5306a92295e04bd4a46d7fdfce71db73, role_id=9c6920b8e6f744a6a80371b4218605fd)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:55:59.474368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e094202-a649-4c4e-8a5b-015a7ce32ded None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:55:59.490932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:55:59.499093 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 206/402] 10.222.0.36 () {58 vars in 1442 bytes} [Fri Sep 2 08:55:59 2022] PUT /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4/groups/5306a92295e04bd4a46d7fdfce71db73/roles/9c6920b8e6f744a6a80371b4218605fd => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:55:59.500272 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c57dc2-2fb1-4794-9134-40e0f5e4a364 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:59.500601 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c57dc2-2fb1-4794-9134-40e0f5e4a364 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:59.500810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c57dc2-2fb1-4794-9134-40e0f5e4a364 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:59.502274 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 207/403] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:55:59.509849 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6568a80-a0f8-468b-b6bb-973ce19e9024 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:55:59.510254 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6568a80-a0f8-468b-b6bb-973ce19e9024 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:55:59.510604 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6568a80-a0f8-468b-b6bb-973ce19e9024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:55:59.540785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-f6568a80-a0f8-468b-b6bb-973ce19e9024 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:55:59.541679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 197/404] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:55:59.542055 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f6568a80-a0f8-468b-b6bb-973ce19e9024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:55:59.648360 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 208/405] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:55:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:55:59.875913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8a857-43d1-4b58-8d75-7d2044948de8 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:59.943140 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6568a80-a0f8-468b-b6bb-973ce19e9024 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:55:59.995224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:00.023859 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8a857-43d1-4b58-8d75-7d2044948de8 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:00.073450 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6568a80-a0f8-468b-b6bb-973ce19e9024 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:00.106422 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:00.111411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c9a9c-e350-41de-9d52-938acf667aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:00.112074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c9a9c-e350-41de-9d52-938acf667aae None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:00.112350 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c9a9c-e350-41de-9d52-938acf667aae None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:00.112623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c9a9c-e350-41de-9d52-938acf667aae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:00.112827 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0271c6-310d-4469-82a5-be08b5dd6fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:00.113615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0271c6-310d-4469-82a5-be08b5dd6fad None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:00.113965 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0271c6-310d-4469-82a5-be08b5dd6fad None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:00.114327 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0271c6-310d-4469-82a5-be08b5dd6fad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:00.126470 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0271c6-310d-4469-82a5-be08b5dd6fad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:00.131668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-5b0271c6-310d-4469-82a5-be08b5dd6fad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:56:00.131668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:00.131668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:00.131668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:00.131668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:00.131668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:00.131668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:00.131668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:00.131668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:00.131668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:00.131668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:00.131668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:00.131668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:00.131668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:00.131668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:00.131668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:00.133515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:00.133515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:00.133515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:00.133515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:00.133515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:00.133515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:00.133515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:00.133515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:00.133515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:00.133515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:00.133515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:00.133515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:00.133515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:00.133515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:00.133515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:00.133515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:00.135564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:00.135564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:00.135564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:00.135564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:00.135564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:00.135564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:00.135564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:00.135564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:00.135564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:00.135564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:00.135564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:56:00.135564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:00.135564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 209/406] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:55:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:00.140883 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0271c6-310d-4469-82a5-be08b5dd6fad None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:00.167713 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c9a9c-e350-41de-9d52-938acf667aae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:00.172639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-967c9a9c-e350-41de-9d52-938acf667aae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:56:00.172639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:00.172639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:00.172639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:00.172639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:00.172639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:00.172639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:00.172639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:00.172639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:00.172639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:00.172639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:00.172639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:00.172639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:00.172639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:00.172639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:00.172639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:00.174745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:00.174745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:00.174745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:00.174745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:00.174745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:00.174745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:00.174745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:00.174745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:00.174745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:00.174745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:00.174745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:00.174745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:00.174745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:00.174745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:00.174745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:00.174745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:00.176773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:00.176773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:00.176773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:00.176773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:00.176773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:00.176773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:00.176773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:00.176773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:00.176773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:00.176773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:00.176773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:56:00.176773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:00.176773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 198/407] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:55:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:00.184017 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c9a9c-e350-41de-9d52-938acf667aae None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:00.250027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0271c6-310d-4469-82a5-be08b5dd6fad None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:00.289004 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10aba4d-2806-4535-b153-16bbdc72dc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:00.289627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aba4d-2806-4535-b153-16bbdc72dc5a None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:00.289907 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aba4d-2806-4535-b153-16bbdc72dc5a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:00.290181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10aba4d-2806-4535-b153-16bbdc72dc5a None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:00.290843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10aba4d-2806-4535-b153-16bbdc72dc5a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:00.291185 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10aba4d-2806-4535-b153-16bbdc72dc5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:00.292567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10aba4d-2806-4535-b153-16bbdc72dc5a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:00.301577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 210/408] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:00.307842 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2698b30-34ae-4a5b-a9dd-dc980ed1cbba None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:00.308129 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2698b30-34ae-4a5b-a9dd-dc980ed1cbba None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:00.308410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2698b30-34ae-4a5b-a9dd-dc980ed1cbba None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:00.309798 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 211/409] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:00.310605 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2698b30-34ae-4a5b-a9dd-dc980ed1cbba None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:00.365363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c9a9c-e350-41de-9d52-938acf667aae None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:00.373072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:00.411036 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93f26a-1391-4f8a-a7fc-de31ff9b9d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:00.412749 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93f26a-1391-4f8a-a7fc-de31ff9b9d3d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:00.413209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93f26a-1391-4f8a-a7fc-de31ff9b9d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:00.413660 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93f26a-1391-4f8a-a7fc-de31ff9b9d3d None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:00.414498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93f26a-1391-4f8a-a7fc-de31ff9b9d3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:00.415487 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93f26a-1391-4f8a-a7fc-de31ff9b9d3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:00.417319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93f26a-1391-4f8a-a7fc-de31ff9b9d3d None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:00.428367 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2698b30-34ae-4a5b-a9dd-dc980ed1cbba None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:00.428629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 199/410] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:56:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:00.430232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515ede6-f565-4b50-8652-08b764e94410 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:00.431104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515ede6-f565-4b50-8652-08b764e94410 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:00.431778 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515ede6-f565-4b50-8652-08b764e94410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:00.446311 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:00.454029 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-d515ede6-f565-4b50-8652-08b764e94410 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:00.455844 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-d515ede6-f565-4b50-8652-08b764e94410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:00.466477 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada67137-22be-40c5-9768-e95b50b8591c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:00.467473 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada67137-22be-40c5-9768-e95b50b8591c None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:00.468020 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada67137-22be-40c5-9768-e95b50b8591c None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:00.468586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada67137-22be-40c5-9768-e95b50b8591c None admin] PATH_INFO: `/v3/groups/5306a92295e04bd4a46d7fdfce71db73` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:00.469933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada67137-22be-40c5-9768-e95b50b8591c None admin] RBAC: Authorizing `identity:get_group(group_id=5306a92295e04bd4a46d7fdfce71db73)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:00.471724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada67137-22be-40c5-9768-e95b50b8591c None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:00.478558 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 212/411] 10.222.0.36 () {56 vars in 1180 bytes} [Fri Sep 2 08:56:00 2022] GET /identity/v3/groups/5306a92295e04bd4a46d7fdfce71db73 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:00.479646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada67137-22be-40c5-9768-e95b50b8591c None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:00.538863 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 200/412] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:00.540162 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d515ede6-f565-4b50-8652-08b764e94410 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:00.604562 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada67137-22be-40c5-9768-e95b50b8591c None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:00.635112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:00.654930 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445bb2b4-8467-4efa-a4fe-5356e25b3089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:00.655667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bb2b4-8467-4efa-a4fe-5356e25b3089 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:00.656639 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bb2b4-8467-4efa-a4fe-5356e25b3089 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:00.656639 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bb2b4-8467-4efa-a4fe-5356e25b3089 None admin] PATH_INFO: `/v3/groups/9f4e02017dac436eae4c8264d70ddfb1` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:00.660118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445bb2b4-8467-4efa-a4fe-5356e25b3089 None admin] RBAC: Authorizing `identity:get_group(group_id=9f4e02017dac436eae4c8264d70ddfb1)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:00.662162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445bb2b4-8467-4efa-a4fe-5356e25b3089 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:00.663684 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 213/413] 10.222.0.36 () {56 vars in 1180 bytes} [Fri Sep 2 08:56:00 2022] GET /identity/v3/groups/9f4e02017dac436eae4c8264d70ddfb1 => generated 229 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:00.665711 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d515ede6-f565-4b50-8652-08b764e94410 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:00.674032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445bb2b4-8467-4efa-a4fe-5356e25b3089 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:00.752973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946dc38f-84c9-41ee-af2e-01128f630327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:00.753625 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946dc38f-84c9-41ee-af2e-01128f630327 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:00.753903 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946dc38f-84c9-41ee-af2e-01128f630327 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:00.754171 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946dc38f-84c9-41ee-af2e-01128f630327 None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:00.759788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946dc38f-84c9-41ee-af2e-01128f630327 None admin] RBAC: Authorizing `identity:get_project(project_id=78733ddb1b714faf83957113a15c51bc)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:00.761238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946dc38f-84c9-41ee-af2e-01128f630327 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:00.763235 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 201/414] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:56:00 2022] GET /identity/v3/projects/78733ddb1b714faf83957113a15c51bc => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:00.768849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946dc38f-84c9-41ee-af2e-01128f630327 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:00.783454 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445bb2b4-8467-4efa-a4fe-5356e25b3089 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:00.789234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:00.822908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11354ec-4a6c-489b-b7a9-1f271efee43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:00.823581 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11354ec-4a6c-489b-b7a9-1f271efee43f None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:00.823855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11354ec-4a6c-489b-b7a9-1f271efee43f None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:00.824123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11354ec-4a6c-489b-b7a9-1f271efee43f None admin] PATH_INFO: `/v3/projects/32923700004a4c11b8926f21bae6d460` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:00.825234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11354ec-4a6c-489b-b7a9-1f271efee43f None admin] RBAC: Authorizing `identity:get_project(project_id=32923700004a4c11b8926f21bae6d460)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:00.826681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11354ec-4a6c-489b-b7a9-1f271efee43f None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:00.828894 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 214/415] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:56:00 2022] GET /identity/v3/projects/32923700004a4c11b8926f21bae6d460 => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:00.842270 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11354ec-4a6c-489b-b7a9-1f271efee43f None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:00.918737 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946dc38f-84c9-41ee-af2e-01128f630327 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:00.952434 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11354ec-4a6c-489b-b7a9-1f271efee43f None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:00.953436 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42838c60-17d4-4881-8825-2ab82238e3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:00.954097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42838c60-17d4-4881-8825-2ab82238e3b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:00.954376 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42838c60-17d4-4881-8825-2ab82238e3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:00.954642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42838c60-17d4-4881-8825-2ab82238e3b2 None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc/groups/5306a92295e04bd4a46d7fdfce71db73/roles/9c6920b8e6f744a6a80371b4218605fd` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:00.955136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:00.957102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42838c60-17d4-4881-8825-2ab82238e3b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=78733ddb1b714faf83957113a15c51bc, group_id=5306a92295e04bd4a46d7fdfce71db73, role_id=9c6920b8e6f744a6a80371b4218605fd)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:00.958571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42838c60-17d4-4881-8825-2ab82238e3b2 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:00.973428 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 202/416] 10.222.0.36 () {58 vars in 1442 bytes} [Fri Sep 2 08:56:00 2022] PUT /identity/v3/projects/78733ddb1b714faf83957113a15c51bc/groups/5306a92295e04bd4a46d7fdfce71db73/roles/9c6920b8e6f744a6a80371b4218605fd => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:56:00.974940 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42838c60-17d4-4881-8825-2ab82238e3b2 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:01.000668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38acee-3115-4a6f-b2ab-6f989595900f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:01.002154 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38acee-3115-4a6f-b2ab-6f989595900f None admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:01.002432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38acee-3115-4a6f-b2ab-6f989595900f None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:01.002704 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa38acee-3115-4a6f-b2ab-6f989595900f None admin] PATH_INFO: `/v3/projects/32923700004a4c11b8926f21bae6d460/groups/9f4e02017dac436eae4c8264d70ddfb1/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:01.005539 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa38acee-3115-4a6f-b2ab-6f989595900f None admin] RBAC: Authorizing `identity:create_grant(project_id=32923700004a4c11b8926f21bae6d460, group_id=9f4e02017dac436eae4c8264d70ddfb1, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:01.007933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa38acee-3115-4a6f-b2ab-6f989595900f None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:01.026586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 215/417] 10.222.0.36 () {58 vars in 1442 bytes} [Fri Sep 2 08:56:00 2022] PUT /identity/v3/projects/32923700004a4c11b8926f21bae6d460/groups/9f4e02017dac436eae4c8264d70ddfb1/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 08:56:01.067963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:01.115039 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42838c60-17d4-4881-8825-2ab82238e3b2 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:01.168894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a22fe-eeb5-4831-a8f5-9e4cc2b326c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:01.169466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a22fe-eeb5-4831-a8f5-9e4cc2b326c4 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:01.169746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a22fe-eeb5-4831-a8f5-9e4cc2b326c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:01.170019 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a22fe-eeb5-4831-a8f5-9e4cc2b326c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:01.186949 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a22fe-eeb5-4831-a8f5-9e4cc2b326c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:01.192527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-675a22fe-eeb5-4831-a8f5-9e4cc2b326c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:56:01.192527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:01.192527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:01.192527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:01.192527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:01.192527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:01.192527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:01.192527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:01.192527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:01.192527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:01.192527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:01.192527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:01.192527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:01.192527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:01.192527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:01.192527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:01.193452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:01.193452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:01.193452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:01.193452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:01.193452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:01.193452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:01.193452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:01.193452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:01.193452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:01.193452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:01.193452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:01.193452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:01.193452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:01.193452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:01.193452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:01.193452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:01.194244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:01.194244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:01.194244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:01.194244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:01.194244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:01.194244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:01.194244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:01.194244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:01.194244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:01.194244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:01.194244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:56:01.194244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:01.194244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 203/418] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:56:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:01.201334 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38acee-3115-4a6f-b2ab-6f989595900f None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:01.326712 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa38acee-3115-4a6f-b2ab-6f989595900f None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:01.344411 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:01.369566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a75f9-7300-4516-a3c8-48f5ebe1674a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:01.371309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a75f9-7300-4516-a3c8-48f5ebe1674a None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:01.371309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a75f9-7300-4516-a3c8-48f5ebe1674a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:01.371309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a75f9-7300-4516-a3c8-48f5ebe1674a None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:01.371585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a75f9-7300-4516-a3c8-48f5ebe1674a None admin] RBAC: Adding query filter params (name=member) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:01.371710 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a75f9-7300-4516-a3c8-48f5ebe1674a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:01.373085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a75f9-7300-4516-a3c8-48f5ebe1674a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:01.391940 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 216/419] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:56:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:01.398157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a22fe-eeb5-4831-a8f5-9e4cc2b326c4 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:01.530360 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a22fe-eeb5-4831-a8f5-9e4cc2b326c4 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:01.555531 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:01.566679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd54fd5-6137-4984-b0a4-61dc31d68b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:01.567358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd54fd5-6137-4984-b0a4-61dc31d68b3c None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:01.567643 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd54fd5-6137-4984-b0a4-61dc31d68b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:01.567969 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd54fd5-6137-4984-b0a4-61dc31d68b3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:01.639578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd54fd5-6137-4984-b0a4-61dc31d68b3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:01.641460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd54fd5-6137-4984-b0a4-61dc31d68b3c None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:01.744189 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 204/420] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:56:01.749841 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a75f9-7300-4516-a3c8-48f5ebe1674a None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:01.845467 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea8e66-44f6-43d4-b06e-40ce62a4a017 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:01.846067 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea8e66-44f6-43d4-b06e-40ce62a4a017 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:01.846532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea8e66-44f6-43d4-b06e-40ce62a4a017 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:01.848138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 205/421] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:56:01.853523 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576629ed-92a4-4a2d-b8e0-089eb17cd1cf None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:01.853807 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576629ed-92a4-4a2d-b8e0-089eb17cd1cf None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:01.854078 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576629ed-92a4-4a2d-b8e0-089eb17cd1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:01.872511 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-576629ed-92a4-4a2d-b8e0-089eb17cd1cf None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:01.873690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-576629ed-92a4-4a2d-b8e0-089eb17cd1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:01.893153 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:01.901877 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a75f9-7300-4516-a3c8-48f5ebe1674a None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:01.944393 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ac0c4-808a-4191-9619-360b50772ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:01.945269 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ac0c4-808a-4191-9619-360b50772ca2 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:01.945545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ac0c4-808a-4191-9619-360b50772ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:01.946192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ac0c4-808a-4191-9619-360b50772ca2 None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:01.947341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ac0c4-808a-4191-9619-360b50772ca2 None admin] RBAC: Authorizing `identity:get_project(project_id=2d3731a0d0214a2ab2046a5d36d29aa4)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:01.949335 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ac0c4-808a-4191-9619-360b50772ca2 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:01.951841 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 217/422] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:56:01 2022] GET /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:01.955767 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40bd19f-a308-4a05-b03a-5686a7834e66 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:01.956050 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40bd19f-a308-4a05-b03a-5686a7834e66 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:01.956324 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40bd19f-a308-4a05-b03a-5686a7834e66 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:01.957640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 218/423] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:01.958890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40bd19f-a308-4a05-b03a-5686a7834e66 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:01.977463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 206/424] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:56:01.979146 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4811c7-4ef5-4510-8c76-f18985c64fdf None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:01.979438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4811c7-4ef5-4510-8c76-f18985c64fdf None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:01.979708 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4811c7-4ef5-4510-8c76-f18985c64fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:02.012967 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-4a4811c7-4ef5-4510-8c76-f18985c64fdf None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:02.014163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-4a4811c7-4ef5-4510-8c76-f18985c64fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:02.042283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:02.133456 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 207/425] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:02.136219 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40bd19f-a308-4a05-b03a-5686a7834e66 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:02.182689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3daf66-be4a-42ab-87bd-2dea98018b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:02.183352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3daf66-be4a-42ab-87bd-2dea98018b32 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:02.183632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3daf66-be4a-42ab-87bd-2dea98018b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:02.183905 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3daf66-be4a-42ab-87bd-2dea98018b32 None admin] PATH_INFO: `/v3/groups/5306a92295e04bd4a46d7fdfce71db73` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:02.185265 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3daf66-be4a-42ab-87bd-2dea98018b32 None admin] RBAC: Authorizing `identity:get_group(group_id=5306a92295e04bd4a46d7fdfce71db73)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:02.186705 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3daf66-be4a-42ab-87bd-2dea98018b32 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:02.188520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 219/426] 10.222.0.36 () {56 vars in 1180 bytes} [Fri Sep 2 08:56:01 2022] GET /identity/v3/groups/5306a92295e04bd4a46d7fdfce71db73 => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:02.194704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4811c7-4ef5-4510-8c76-f18985c64fdf None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:02.316626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:02.335630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4811c7-4ef5-4510-8c76-f18985c64fdf None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:02.377451 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c7a12-bec0-4aa8-930c-4d9cf195d815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:02.378077 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c7a12-bec0-4aa8-930c-4d9cf195d815 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:02.378359 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c7a12-bec0-4aa8-930c-4d9cf195d815 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:02.378633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c7a12-bec0-4aa8-930c-4d9cf195d815 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:02.379416 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c7a12-bec0-4aa8-930c-4d9cf195d815 None admin] RBAC: Adding query filter params (group.id=5306a92295e04bd4a46d7fdfce71db73, role.id=08c2a54a85ef4bb9892a5769ef8a5d95, scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:02.379728 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c7a12-bec0-4aa8-930c-4d9cf195d815 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:02.381735 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c7a12-bec0-4aa8-930c-4d9cf195d815 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:02.388926 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 208/427] 10.222.0.36 () {56 vars in 1376 bytes} [Fri Sep 2 08:56:02 2022] GET /identity/v3/role_assignments?group.id=5306a92295e04bd4a46d7fdfce71db73&scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4&role.id=08c2a54a85ef4bb9892a5769ef8a5d95 => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:02.484917 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3daf66-be4a-42ab-87bd-2dea98018b32 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:02.530325 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c7a12-bec0-4aa8-930c-4d9cf195d815 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:02.620132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3daf66-be4a-42ab-87bd-2dea98018b32 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:02.638742 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:02.667602 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9860b-c1a7-4e20-9dab-b4c2c3829934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:02.668838 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9860b-c1a7-4e20-9dab-b4c2c3829934 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:02.669339 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9860b-c1a7-4e20-9dab-b4c2c3829934 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:02.669884 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b9860b-c1a7-4e20-9dab-b4c2c3829934 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:02.670862 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c7a12-bec0-4aa8-930c-4d9cf195d815 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:02.684322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b9860b-c1a7-4e20-9dab-b4c2c3829934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:02.691485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-85b9860b-c1a7-4e20-9dab-b4c2c3829934 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:56:02.691485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:02.691485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:02.691485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:02.691485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:02.691485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:02.691485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:02.691485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:02.691485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:02.691485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:02.691485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:02.691485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:02.691485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:02.691485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:02.691485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:02.691485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:02.692957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:02.692957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:02.692957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:02.692957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:02.692957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:02.692957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:02.692957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:02.692957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:02.692957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:02.692957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:02.692957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:02.692957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:02.692957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:02.692957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:02.692957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:02.692957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:02.693775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:02.693775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:02.693775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:02.693775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:02.693775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:02.693775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:02.693775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:02.693775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:02.693775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:02.693775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:02.693775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:56:02.693775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:02.696495 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 220/428] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:56:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:02.701823 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9860b-c1a7-4e20-9dab-b4c2c3829934 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:02.713677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8dc93-a00a-4309-8f60-3a44a60069c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:02.714464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8dc93-a00a-4309-8f60-3a44a60069c3 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:02.714848 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8dc93-a00a-4309-8f60-3a44a60069c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:02.715353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8dc93-a00a-4309-8f60-3a44a60069c3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:02.730073 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a8dc93-a00a-4309-8f60-3a44a60069c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:02.735654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-52a8dc93-a00a-4309-8f60-3a44a60069c3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:56:02.735654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:02.735654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:02.735654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:02.735654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:02.735654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:02.735654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:02.735654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:02.735654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:02.735654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:02.735654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:02.735654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:02.735654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:02.735654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:02.735654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:02.735654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:02.736831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:02.736831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:02.736831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:02.736831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:02.736831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:02.736831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:02.736831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:02.736831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:02.736831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:02.736831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:02.736831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:02.736831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:02.736831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:02.736831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:02.736831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:02.736831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:02.737811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:02.737811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:02.737811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:02.737811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:02.737811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:02.737811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:02.737811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:02.737811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:02.737811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:02.737811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:02.737811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 08:56:02.737811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:02.739056 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 209/429] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:56:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:02.746172 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8dc93-a00a-4309-8f60-3a44a60069c3 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:02.762058 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:02.829194 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b9860b-c1a7-4e20-9dab-b4c2c3829934 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:02.871351 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8dc93-a00a-4309-8f60-3a44a60069c3 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:02.876346 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541aa5c6-26c5-490b-8f97-ef06f0ef680b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:02.876959 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541aa5c6-26c5-490b-8f97-ef06f0ef680b None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:02.877243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541aa5c6-26c5-490b-8f97-ef06f0ef680b None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:02.877514 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541aa5c6-26c5-490b-8f97-ef06f0ef680b None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:02.878177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541aa5c6-26c5-490b-8f97-ef06f0ef680b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:02.878475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541aa5c6-26c5-490b-8f97-ef06f0ef680b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:02.879922 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541aa5c6-26c5-490b-8f97-ef06f0ef680b None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:02.891050 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 221/430] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:02.897008 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541aa5c6-26c5-490b-8f97-ef06f0ef680b None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:02.916040 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97c016-b849-44f6-bacb-0d356916c851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:02.917720 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97c016-b849-44f6-bacb-0d356916c851 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:02.918237 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97c016-b849-44f6-bacb-0d356916c851 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:02.918716 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97c016-b849-44f6-bacb-0d356916c851 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:02.919633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97c016-b849-44f6-bacb-0d356916c851 None admin] RBAC: Adding query filter params (name=member) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:02.920284 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97c016-b849-44f6-bacb-0d356916c851 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:02.922086 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97c016-b849-44f6-bacb-0d356916c851 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:02.931939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 210/431] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:56:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:02.938143 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97c016-b849-44f6-bacb-0d356916c851 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:02.942885 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:03.036188 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541aa5c6-26c5-490b-8f97-ef06f0ef680b None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:03.037373 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:03.086652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97c016-b849-44f6-bacb-0d356916c851 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:03.092956 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad7bf8-0056-4581-ab58-711997dfbb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:03.093586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad7bf8-0056-4581-ab58-711997dfbb14 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:03.093898 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad7bf8-0056-4581-ab58-711997dfbb14 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:03.094178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad7bf8-0056-4581-ab58-711997dfbb14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:03.136142 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf23771-d6ff-4020-bc25-6424dfcdfc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:03.136556 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf23771-d6ff-4020-bc25-6424dfcdfc5a None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:03.136835 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf23771-d6ff-4020-bc25-6424dfcdfc5a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:03.137105 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf23771-d6ff-4020-bc25-6424dfcdfc5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:03.174317 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad7bf8-0056-4581-ab58-711997dfbb14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:03.176222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad7bf8-0056-4581-ab58-711997dfbb14 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:03.213606 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:03.215092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf23771-d6ff-4020-bc25-6424dfcdfc5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:03.216775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf23771-d6ff-4020-bc25-6424dfcdfc5a None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:03.288244 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 222/432] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:56:03.290975 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:03.295745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad7bf8-0056-4581-ab58-711997dfbb14 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:03.330063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 211/433] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:56:03.338674 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf23771-d6ff-4020-bc25-6424dfcdfc5a None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:03.420600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad7bf8-0056-4581-ab58-711997dfbb14 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:03.444249 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf23771-d6ff-4020-bc25-6424dfcdfc5a None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:03.456326 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c319cc31-9c5d-4ab9-acab-b3700170c4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:03.457059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319cc31-9c5d-4ab9-acab-b3700170c4d4 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:03.457330 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319cc31-9c5d-4ab9-acab-b3700170c4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:03.457693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c319cc31-9c5d-4ab9-acab-b3700170c4d4 None admin] PATH_INFO: `/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:03.458824 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319cc31-9c5d-4ab9-acab-b3700170c4d4 None admin] RBAC: Authorizing `identity:get_project(project_id=2d3731a0d0214a2ab2046a5d36d29aa4)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:03.460288 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c319cc31-9c5d-4ab9-acab-b3700170c4d4 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:03.466366 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 223/434] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:56:03 2022] GET /identity/v3/projects/2d3731a0d0214a2ab2046a5d36d29aa4 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:03.466366 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7e2d7-cab9-42dc-9b52-93f2faf885c7 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:03.466366 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7e2d7-cab9-42dc-9b52-93f2faf885c7 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:03.466366 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7e2d7-cab9-42dc-9b52-93f2faf885c7 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:03.466366 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 224/435] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:03.470054 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7e2d7-cab9-42dc-9b52-93f2faf885c7 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:03.473961 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:03.481883 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d49a0-6043-44fe-920c-4d96f8c2dac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:03.482519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d49a0-6043-44fe-920c-4d96f8c2dac0 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:03.482801 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d49a0-6043-44fe-920c-4d96f8c2dac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:03.483104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d49a0-6043-44fe-920c-4d96f8c2dac0 None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:03.484238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d49a0-6043-44fe-920c-4d96f8c2dac0 None admin] RBAC: Authorizing `identity:get_project(project_id=78733ddb1b714faf83957113a15c51bc)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:03.485665 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d49a0-6043-44fe-920c-4d96f8c2dac0 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:03.487325 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 212/436] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:56:03 2022] GET /identity/v3/projects/78733ddb1b714faf83957113a15c51bc => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:03.488478 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7acbaa-9095-4b7b-9b0d-369b1407962d None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:03.488762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7acbaa-9095-4b7b-9b0d-369b1407962d None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:03.489536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7acbaa-9095-4b7b-9b0d-369b1407962d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:03.507446 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-bd7acbaa-9095-4b7b-9b0d-369b1407962d None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:03.508642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-bd7acbaa-9095-4b7b-9b0d-369b1407962d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:03.538750 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:03.581738 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7e2d7-cab9-42dc-9b52-93f2faf885c7 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:03.585095 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 213/437] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:03.585985 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7acbaa-9095-4b7b-9b0d-369b1407962d None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:03.628183 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556b2f1-fe6c-4241-9214-187a43dfebcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:03.628867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556b2f1-fe6c-4241-9214-187a43dfebcd None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:03.629147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556b2f1-fe6c-4241-9214-187a43dfebcd None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:03.629415 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556b2f1-fe6c-4241-9214-187a43dfebcd None admin] PATH_INFO: `/v3/groups/5306a92295e04bd4a46d7fdfce71db73` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:03.631222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e556b2f1-fe6c-4241-9214-187a43dfebcd None admin] RBAC: Authorizing `identity:get_group(group_id=5306a92295e04bd4a46d7fdfce71db73)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:03.632768 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e556b2f1-fe6c-4241-9214-187a43dfebcd None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:03.634578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 225/438] 10.222.0.36 () {56 vars in 1180 bytes} [Fri Sep 2 08:56:03 2022] GET /identity/v3/groups/5306a92295e04bd4a46d7fdfce71db73 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:03.640261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066679b-3522-4fd5-99a3-e2ca8caf61c2 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:03.640536 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066679b-3522-4fd5-99a3-e2ca8caf61c2 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:03.640802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066679b-3522-4fd5-99a3-e2ca8caf61c2 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:03.643237 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 226/439] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:03.643237 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066679b-3522-4fd5-99a3-e2ca8caf61c2 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:03.690318 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7acbaa-9095-4b7b-9b0d-369b1407962d None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:03.721107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:03.728498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b860179-4668-4a89-ae1c-1baba7cc9a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:03.729487 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b860179-4668-4a89-ae1c-1baba7cc9a9a None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:03.729780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b860179-4668-4a89-ae1c-1baba7cc9a9a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:03.730035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b860179-4668-4a89-ae1c-1baba7cc9a9a None admin] PATH_INFO: `/v3/groups/5306a92295e04bd4a46d7fdfce71db73` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:03.731485 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b860179-4668-4a89-ae1c-1baba7cc9a9a None admin] RBAC: Authorizing `identity:get_group(group_id=5306a92295e04bd4a46d7fdfce71db73)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:03.732919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b860179-4668-4a89-ae1c-1baba7cc9a9a None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:03.734467 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 214/440] 10.222.0.36 () {56 vars in 1180 bytes} [Fri Sep 2 08:56:03 2022] GET /identity/v3/groups/5306a92295e04bd4a46d7fdfce71db73 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:03.739697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9feb2-be54-4aa1-93a0-5442ab293e31 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:03.739697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9feb2-be54-4aa1-93a0-5442ab293e31 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:03.739697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9feb2-be54-4aa1-93a0-5442ab293e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:03.754277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-18e9feb2-be54-4aa1-93a0-5442ab293e31 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:03.755537 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-18e9feb2-be54-4aa1-93a0-5442ab293e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:03.758600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066679b-3522-4fd5-99a3-e2ca8caf61c2 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:03.784014 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:03.793003 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067225a5-17d7-4621-9330-fa44ad1cf42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:03.793604 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067225a5-17d7-4621-9330-fa44ad1cf42f None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:03.793881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067225a5-17d7-4621-9330-fa44ad1cf42f None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:03.794152 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067225a5-17d7-4621-9330-fa44ad1cf42f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:03.794867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067225a5-17d7-4621-9330-fa44ad1cf42f None admin] RBAC: Adding query filter params (group.id=5306a92295e04bd4a46d7fdfce71db73, role.id=9c6920b8e6f744a6a80371b4218605fd, scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:03.795200 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067225a5-17d7-4621-9330-fa44ad1cf42f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:03.796626 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067225a5-17d7-4621-9330-fa44ad1cf42f None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:03.803136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 227/441] 10.222.0.36 () {56 vars in 1376 bytes} [Fri Sep 2 08:56:03 2022] GET /identity/v3/role_assignments?group.id=5306a92295e04bd4a46d7fdfce71db73&scope.project.id=2d3731a0d0214a2ab2046a5d36d29aa4&role.id=9c6920b8e6f744a6a80371b4218605fd => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:03.804201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067225a5-17d7-4621-9330-fa44ad1cf42f None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:03.850013 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 215/442] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:03.855504 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9feb2-be54-4aa1-93a0-5442ab293e31 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:03.938750 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067225a5-17d7-4621-9330-fa44ad1cf42f None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:03.977525 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204f112-d144-4c67-9333-8b2f07803498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:03.978166 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204f112-d144-4c67-9333-8b2f07803498 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:03.978452 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204f112-d144-4c67-9333-8b2f07803498 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:03.978725 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6204f112-d144-4c67-9333-8b2f07803498 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:03.979477 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204f112-d144-4c67-9333-8b2f07803498 None admin] RBAC: Adding query filter params (group.id=5306a92295e04bd4a46d7fdfce71db73, role.id=08c2a54a85ef4bb9892a5769ef8a5d95, scope.project.id=78733ddb1b714faf83957113a15c51bc) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:03.979784 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204f112-d144-4c67-9333-8b2f07803498 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:03.981229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6204f112-d144-4c67-9333-8b2f07803498 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:03.982909 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9feb2-be54-4aa1-93a0-5442ab293e31 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:03.984177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:03.988817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 228/443] 10.222.0.36 () {56 vars in 1376 bytes} [Fri Sep 2 08:56:03 2022] GET /identity/v3/role_assignments?group.id=5306a92295e04bd4a46d7fdfce71db73&scope.project.id=78733ddb1b714faf83957113a15c51bc&role.id=08c2a54a85ef4bb9892a5769ef8a5d95 => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:04.028230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7ec21-ea4d-4620-9e23-ac9883318bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:04.029244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7ec21-ea4d-4620-9e23-ac9883318bdb None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:04.029581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7ec21-ea4d-4620-9e23-ac9883318bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:04.029918 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7ec21-ea4d-4620-9e23-ac9883318bdb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:04.042499 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7ec21-ea4d-4620-9e23-ac9883318bdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:04.047630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-86d7ec21-ea4d-4620-9e23-ac9883318bdb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:56:04.047630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:04.047630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:04.047630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:04.047630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:04.047630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:04.047630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:04.047630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:04.047630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:04.047630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:04.047630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:04.047630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:04.047630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:04.047630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:04.047630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:04.047630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:04.048589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:04.048589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:04.048589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:04.048589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:04.048589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:04.048589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:04.048589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:04.048589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:04.048589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:04.048589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:04.048589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:04.048589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:04.048589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:04.048589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:04.048589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:04.048589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:04.049390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:04.049390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:04.049390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:04.049390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:04.049390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:04.049390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:04.049390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:04.049390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:04.049390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:04.049390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:04.049390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:56:04.049390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:04.053680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 216/444] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:56:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:04.055999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204f112-d144-4c67-9333-8b2f07803498 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:04.120458 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7ec21-ea4d-4620-9e23-ac9883318bdb None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:04.156822 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:04.173321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6204f112-d144-4c67-9333-8b2f07803498 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:04.220178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc951b-0a08-4b1c-bdeb-7d7adc38d006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:04.220798 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc951b-0a08-4b1c-bdeb-7d7adc38d006 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:04.221082 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc951b-0a08-4b1c-bdeb-7d7adc38d006 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:04.221351 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfc951b-0a08-4b1c-bdeb-7d7adc38d006 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:04.222020 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc951b-0a08-4b1c-bdeb-7d7adc38d006 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:04.222323 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc951b-0a08-4b1c-bdeb-7d7adc38d006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:04.224283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfc951b-0a08-4b1c-bdeb-7d7adc38d006 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:04.233357 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 229/445] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:56:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:04.238459 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc951b-0a08-4b1c-bdeb-7d7adc38d006 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:04.254699 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7ec21-ea4d-4620-9e23-ac9883318bdb None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:04.294641 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0cca89-7b3b-4349-b721-a4f362c60358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:04.295433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0cca89-7b3b-4349-b721-a4f362c60358 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:04.295697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0cca89-7b3b-4349-b721-a4f362c60358 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:04.296021 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0cca89-7b3b-4349-b721-a4f362c60358 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:04.308669 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0cca89-7b3b-4349-b721-a4f362c60358 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:04.313747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-3f0cca89-7b3b-4349-b721-a4f362c60358 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:56:04.313747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:04.313747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:04.313747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:04.313747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:04.313747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:04.313747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:04.313747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:04.313747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:04.313747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:04.313747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:04.313747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:04.313747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:04.313747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:04.313747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:04.313747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:04.314646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:04.314646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:04.314646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:04.314646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:04.314646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:04.314646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:04.314646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:04.314646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:04.314646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:04.314646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:04.314646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:04.314646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:04.314646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:04.314646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:04.314646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:04.314646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:04.315789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:04.315789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:04.315789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:04.315789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:04.315789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:04.315789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:04.315789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:04.315789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:04.315789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:04.315789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:04.315789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:56:04.315789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:04.315789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 217/446] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:56:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:04.321089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0cca89-7b3b-4349-b721-a4f362c60358 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:04.323502 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:04.371603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfc951b-0a08-4b1c-bdeb-7d7adc38d006 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:04.414851 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da7a04-3319-4d00-b443-4091739fa7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:04.415605 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da7a04-3319-4d00-b443-4091739fa7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:04.415952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da7a04-3319-4d00-b443-4091739fa7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:04.416227 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da7a04-3319-4d00-b443-4091739fa7e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:04.423415 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:04.445357 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0cca89-7b3b-4349-b721-a4f362c60358 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:04.479395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611f496-57e0-4cc0-90ae-bbc800fcbee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:04.479994 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611f496-57e0-4cc0-90ae-bbc800fcbee9 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:04.480272 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611f496-57e0-4cc0-90ae-bbc800fcbee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:04.480537 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611f496-57e0-4cc0-90ae-bbc800fcbee9 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:04.481192 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8611f496-57e0-4cc0-90ae-bbc800fcbee9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:04.481490 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8611f496-57e0-4cc0-90ae-bbc800fcbee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:04.482846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8611f496-57e0-4cc0-90ae-bbc800fcbee9 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:04.489474 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da7a04-3319-4d00-b443-4091739fa7e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:04.490635 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 218/447] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:04.490914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da7a04-3319-4d00-b443-4091739fa7e2 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:04.496022 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611f496-57e0-4cc0-90ae-bbc800fcbee9 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:04.549915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:04.590365 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 230/448] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 02 08:56:04.594892 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611f496-57e0-4cc0-90ae-bbc800fcbee9 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:04.595987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da7a04-3319-4d00-b443-4091739fa7e2 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:04.629109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee4700-481e-4e4c-937b-bf2c73156e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:04.630218 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee4700-481e-4e4c-937b-bf2c73156e13 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:04.630493 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee4700-481e-4e4c-937b-bf2c73156e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:04.630757 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee4700-481e-4e4c-937b-bf2c73156e13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:04.648053 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:04.696320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da7a04-3319-4d00-b443-4091739fa7e2 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:04.698203 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee4700-481e-4e4c-937b-bf2c73156e13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:04.699695 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee4700-481e-4e4c-937b-bf2c73156e13 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:04.729917 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725cf3b-71d4-4605-ba7b-b6b5a3a565be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:04.730536 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725cf3b-71d4-4605-ba7b-b6b5a3a565be None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:04.730811 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725cf3b-71d4-4605-ba7b-b6b5a3a565be None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:04.731090 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0725cf3b-71d4-4605-ba7b-b6b5a3a565be None admin] PATH_INFO: `/v3/projects/32923700004a4c11b8926f21bae6d460` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:04.732230 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725cf3b-71d4-4605-ba7b-b6b5a3a565be None admin] RBAC: Authorizing `identity:get_project(project_id=32923700004a4c11b8926f21bae6d460)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:04.733647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0725cf3b-71d4-4605-ba7b-b6b5a3a565be None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:04.735273 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 231/449] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:56:04 2022] GET /identity/v3/projects/32923700004a4c11b8926f21bae6d460 => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:04.740853 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725cf3b-71d4-4605-ba7b-b6b5a3a565be None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:04.769820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:04.799758 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 219/450] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 02 08:56:04.805691 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee4700-481e-4e4c-937b-bf2c73156e13 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:04.839833 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0725cf3b-71d4-4605-ba7b-b6b5a3a565be None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:04.851795 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:04.873097 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ade2c-189e-4c2c-ae89-a41708b80756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:04.873714 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ade2c-189e-4c2c-ae89-a41708b80756 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:04.873989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ade2c-189e-4c2c-ae89-a41708b80756 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:04.874256 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ade2c-189e-4c2c-ae89-a41708b80756 None admin] PATH_INFO: `/v3/groups/9f4e02017dac436eae4c8264d70ddfb1` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:04.875392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ade2c-189e-4c2c-ae89-a41708b80756 None admin] RBAC: Authorizing `identity:get_group(group_id=9f4e02017dac436eae4c8264d70ddfb1)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:04.876823 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ade2c-189e-4c2c-ae89-a41708b80756 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:04.878367 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 232/451] 10.222.0.36 () {56 vars in 1180 bytes} [Fri Sep 2 08:56:04 2022] GET /identity/v3/groups/9f4e02017dac436eae4c8264d70ddfb1 => generated 229 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:04.884507 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ade2c-189e-4c2c-ae89-a41708b80756 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:04.902062 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee4700-481e-4e4c-937b-bf2c73156e13 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:04.935132 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfaa8fc-f841-4cc7-b7cb-d920b1c4460b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:04.935753 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfaa8fc-f841-4cc7-b7cb-d920b1c4460b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:04.936028 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfaa8fc-f841-4cc7-b7cb-d920b1c4460b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:04.936294 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfaa8fc-f841-4cc7-b7cb-d920b1c4460b None admin] PATH_INFO: `/v3/projects/78733ddb1b714faf83957113a15c51bc` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:04.937411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfaa8fc-f841-4cc7-b7cb-d920b1c4460b None admin] RBAC: Authorizing `identity:get_project(project_id=78733ddb1b714faf83957113a15c51bc)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:04.938847 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfaa8fc-f841-4cc7-b7cb-d920b1c4460b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:04.940478 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 220/452] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:56:04 2022] GET /identity/v3/projects/78733ddb1b714faf83957113a15c51bc => generated 302 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:04.946483 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfaa8fc-f841-4cc7-b7cb-d920b1c4460b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:04.979978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:04.984141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ade2c-189e-4c2c-ae89-a41708b80756 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:05.018802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161d219-8847-4015-b4b5-3023011c3d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:05.019357 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161d219-8847-4015-b4b5-3023011c3d05 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:05.019630 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161d219-8847-4015-b4b5-3023011c3d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:05.019895 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161d219-8847-4015-b4b5-3023011c3d05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:05.020632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161d219-8847-4015-b4b5-3023011c3d05 None admin] RBAC: Adding query filter params (group.id=9f4e02017dac436eae4c8264d70ddfb1, role.id=a0a76de72fba4036bc1e3e64df717c7f, scope.project.id=32923700004a4c11b8926f21bae6d460) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:05.020941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161d219-8847-4015-b4b5-3023011c3d05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:05.022355 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161d219-8847-4015-b4b5-3023011c3d05 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:05.028812 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 233/453] 10.222.0.36 () {56 vars in 1376 bytes} [Fri Sep 2 08:56:04 2022] GET /identity/v3/role_assignments?group.id=9f4e02017dac436eae4c8264d70ddfb1&scope.project.id=32923700004a4c11b8926f21bae6d460&role.id=a0a76de72fba4036bc1e3e64df717c7f => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:05.044351 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfaa8fc-f841-4cc7-b7cb-d920b1c4460b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:05.063346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:05.082975 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e486729f-1f91-413e-8098-967bbbd7cdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:05.083650 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486729f-1f91-413e-8098-967bbbd7cdf0 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:05.083927 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486729f-1f91-413e-8098-967bbbd7cdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:05.084200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486729f-1f91-413e-8098-967bbbd7cdf0 None admin] PATH_INFO: `/v3/groups/5306a92295e04bd4a46d7fdfce71db73` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:05.085367 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e486729f-1f91-413e-8098-967bbbd7cdf0 None admin] RBAC: Authorizing `identity:get_group(group_id=5306a92295e04bd4a46d7fdfce71db73)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:05.086822 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e486729f-1f91-413e-8098-967bbbd7cdf0 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:05.088476 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 221/454] 10.222.0.36 () {56 vars in 1180 bytes} [Fri Sep 2 08:56:04 2022] GET /identity/v3/groups/5306a92295e04bd4a46d7fdfce71db73 => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:05.094641 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161d219-8847-4015-b4b5-3023011c3d05 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:05.196859 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161d219-8847-4015-b4b5-3023011c3d05 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:05.229613 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e96e2b-0bc1-4008-b3af-d6e7467991f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:05.230217 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e96e2b-0bc1-4008-b3af-d6e7467991f5 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:05.230495 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e96e2b-0bc1-4008-b3af-d6e7467991f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:05.230767 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e96e2b-0bc1-4008-b3af-d6e7467991f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:05.231521 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e96e2b-0bc1-4008-b3af-d6e7467991f5 None admin] RBAC: Adding query filter params (group.id=5306a92295e04bd4a46d7fdfce71db73, role.id=9c6920b8e6f744a6a80371b4218605fd, scope.project.id=78733ddb1b714faf83957113a15c51bc) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:05.231826 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e96e2b-0bc1-4008-b3af-d6e7467991f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:05.233271 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e96e2b-0bc1-4008-b3af-d6e7467991f5 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:05.238796 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 234/455] 10.222.0.36 () {56 vars in 1376 bytes} [Fri Sep 2 08:56:05 2022] GET /identity/v3/role_assignments?group.id=5306a92295e04bd4a46d7fdfce71db73&scope.project.id=78733ddb1b714faf83957113a15c51bc&role.id=9c6920b8e6f744a6a80371b4218605fd => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:07.619659 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3979cb8-1c8d-4c01-b0c2-bf4b5fb8945e None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:07.620154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3979cb8-1c8d-4c01-b0c2-bf4b5fb8945e None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:07.620154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3979cb8-1c8d-4c01-b0c2-bf4b5fb8945e None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:07.621449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 222/456] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:07.627338 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48ab08-eadf-40cc-9093-a61cc2779275 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:07.628141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48ab08-eadf-40cc-9093-a61cc2779275 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:07.628625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48ab08-eadf-40cc-9093-a61cc2779275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:07.652002 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-ad48ab08-eadf-40cc-9093-a61cc2779275 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:07.653493 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-ad48ab08-eadf-40cc-9093-a61cc2779275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:07.679132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:07.741993 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 235/457] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:07.986813 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f85f3-a2a8-4476-a433-0cf872c55b36 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:07.986996 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f85f3-a2a8-4476-a433-0cf872c55b36 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:07.987294 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1f85f3-a2a8-4476-a433-0cf872c55b36 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:07.988935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 223/458] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:07.993732 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f80ec-c311-4220-9fba-3214e34fd439 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:07.994236 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f80ec-c311-4220-9fba-3214e34fd439 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:07.994939 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f80ec-c311-4220-9fba-3214e34fd439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:08.022798 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-6e7f80ec-c311-4220-9fba-3214e34fd439 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:08.024303 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6e7f80ec-c311-4220-9fba-3214e34fd439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:08.067701 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f85f3-a2a8-4476-a433-0cf872c55b36 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:08.116656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 236/459] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:08.219392 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1f85f3-a2a8-4476-a433-0cf872c55b36 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:08.262735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d2a4a-a6cc-4b6e-98ca-e5739bd7b060 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:08.262923 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d2a4a-a6cc-4b6e-98ca-e5739bd7b060 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:08.263231 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d2a4a-a6cc-4b6e-98ca-e5739bd7b060 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:08.264882 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 237/460] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:56:08.267061 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379fef67-d895-4fc3-ad8e-73d4778eedfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:08.268021 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fef67-d895-4fc3-ad8e-73d4778eedfb None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:08.268495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fef67-d895-4fc3-ad8e-73d4778eedfb None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:08.269038 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379fef67-d895-4fc3-ad8e-73d4778eedfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:08.271012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35174a7-e192-43ce-8d08-53346763942f None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:08.271319 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35174a7-e192-43ce-8d08-53346763942f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:08.271739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35174a7-e192-43ce-8d08-53346763942f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:08.286335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379fef67-d895-4fc3-ad8e-73d4778eedfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:08.288310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379fef67-d895-4fc3-ad8e-73d4778eedfb None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:08.303968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-379fef67-d895-4fc3-ad8e-73d4778eedfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:08.303968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:08.303968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:08.303968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:08.303968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:08.303968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:08.303968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:08.303968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:08.303968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:08.303968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:08.303968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:08.303968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:08.303968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:08.303968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:08.303968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:08.303968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:08.304941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:08.304941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:08.304941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:08.304941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:08.304941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:08.304941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:08.304941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:08.304941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:08.304941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:08.304941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:08.304941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:08.304941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:08.304941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:08.304941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:08.304941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:08.304941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:08.304941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:08.305785 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:08.305785 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:08.305785 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:08.307887 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 224/461] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:08.315314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379fef67-d895-4fc3-ad8e-73d4778eedfb None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:08.318879 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-e35174a7-e192-43ce-8d08-53346763942f None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:08.320762 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-e35174a7-e192-43ce-8d08-53346763942f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:08.353345 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:08.425764 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379fef67-d895-4fc3-ad8e-73d4778eedfb None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:08.450586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 238/462] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:08.452452 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c513ff-248c-40f2-9e53-ea00fd3a02bd None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:08.452741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c513ff-248c-40f2-9e53-ea00fd3a02bd None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:08.453425 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c513ff-248c-40f2-9e53-ea00fd3a02bd None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:08.454970 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 239/463] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:08.460636 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4ed28-56ed-46e5-b331-e377033121d0 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:08.460922 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4ed28-56ed-46e5-b331-e377033121d0 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:08.461204 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4ed28-56ed-46e5-b331-e377033121d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:08.465118 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76331345-3136-4fa2-92f0-a8c6b066aa78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:08.465768 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76331345-3136-4fa2-92f0-a8c6b066aa78 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:08.466147 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76331345-3136-4fa2-92f0-a8c6b066aa78 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:08.466423 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76331345-3136-4fa2-92f0-a8c6b066aa78 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:08.467087 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76331345-3136-4fa2-92f0-a8c6b066aa78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:08.467520 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76331345-3136-4fa2-92f0-a8c6b066aa78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:08.468987 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76331345-3136-4fa2-92f0-a8c6b066aa78 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:08.479152 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-57a4ed28-56ed-46e5-b331-e377033121d0 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:08.480332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-57a4ed28-56ed-46e5-b331-e377033121d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:08.481461 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 225/464] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:08.487846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76331345-3136-4fa2-92f0-a8c6b066aa78 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:08.496437 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:08.561849 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 240/465] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:56:08.562830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4ed28-56ed-46e5-b331-e377033121d0 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:08.618774 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:08.627087 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76331345-3136-4fa2-92f0-a8c6b066aa78 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:08.691412 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310eb4f-bc8a-45e4-9549-cde5468d6608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:08.692922 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310eb4f-bc8a-45e4-9549-cde5468d6608 None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:08.694586 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310eb4f-bc8a-45e4-9549-cde5468d6608 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:08.696033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310eb4f-bc8a-45e4-9549-cde5468d6608 None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:08.696899 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4310eb4f-bc8a-45e4-9549-cde5468d6608 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:08.703206 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4310eb4f-bc8a-45e4-9549-cde5468d6608 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:08.749480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 226/466] 10.222.0.36 () {60 vars in 1132 bytes} [Fri Sep 2 08:56:08 2022] POST /identity/v3/users => generated 246 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:56:08.774658 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4ed28-56ed-46e5-b331-e377033121d0 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:08.802680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310eb4f-bc8a-45e4-9549-cde5468d6608 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:08.830546 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74fe2ac-8379-4366-86f6-508cfa3e4e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:08.831601 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74fe2ac-8379-4366-86f6-508cfa3e4e08 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:08.832077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74fe2ac-8379-4366-86f6-508cfa3e4e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:08.832589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74fe2ac-8379-4366-86f6-508cfa3e4e08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:08.855110 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74fe2ac-8379-4366-86f6-508cfa3e4e08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:08.859743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74fe2ac-8379-4366-86f6-508cfa3e4e08 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:08.872205 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:08.883547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-b74fe2ac-8379-4366-86f6-508cfa3e4e08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:08.883547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:08.883547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:08.883547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:08.883547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:08.883547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:08.883547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:08.883547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:08.883547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:08.883547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:08.883547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:08.883547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:08.883547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:08.883547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:08.883547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:08.883547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:08.884529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:08.884529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:08.884529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:08.884529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:08.884529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:08.884529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:08.884529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:08.884529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:08.884529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:08.884529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:08.884529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:08.884529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:08.884529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:08.884529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:08.884529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:08.884529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:08.884529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:08.885380 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:08.885380 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:08.885380 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:08.887241 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 241/467] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:08.895467 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74fe2ac-8379-4366-86f6-508cfa3e4e08 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:08.983675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310eb4f-bc8a-45e4-9549-cde5468d6608 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:09.021915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74fe2ac-8379-4366-86f6-508cfa3e4e08 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:09.028660 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd8133-d065-4283-9367-c5e95fd8ff70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:09.030091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd8133-d065-4283-9367-c5e95fd8ff70 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:09.030571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd8133-d065-4283-9367-c5e95fd8ff70 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:09.031027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd8133-d065-4283-9367-c5e95fd8ff70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:09.045008 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd8133-d065-4283-9367-c5e95fd8ff70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:09.046475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd8133-d065-4283-9367-c5e95fd8ff70 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:09.063035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2ab0c-11d1-44bd-ab24-f8f4fa77f6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:09.063319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-6fcd8133-d065-4283-9367-c5e95fd8ff70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:09.063319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:09.063319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:09.063319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:09.063319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:09.063319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:09.063319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:09.063319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:09.063319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:09.063319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:09.063319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:09.063319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:09.063319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:09.063319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:09.063319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:09.063319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:09.064190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:09.064190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:09.064190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:09.064190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:09.064190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:09.064190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:09.064190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:09.064190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:09.064190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:09.064190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:09.064190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:09.064190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:09.064190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:09.064190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:09.064190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:09.064190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:09.064190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:09.065025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:09.065025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:09.065025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:09.065025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 227/468] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:09.065580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd8133-d065-4283-9367-c5e95fd8ff70 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:09.066270 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2ab0c-11d1-44bd-ab24-f8f4fa77f6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:09.066686 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2ab0c-11d1-44bd-ab24-f8f4fa77f6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:09.067178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb2ab0c-11d1-44bd-ab24-f8f4fa77f6d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:09.067954 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb2ab0c-11d1-44bd-ab24-f8f4fa77f6d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:09.068456 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb2ab0c-11d1-44bd-ab24-f8f4fa77f6d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:09.069159 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:09.070718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb2ab0c-11d1-44bd-ab24-f8f4fa77f6d6 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:09.092237 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 242/469] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:09.093294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2ab0c-11d1-44bd-ab24-f8f4fa77f6d6 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:09.156674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:09.193431 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd8133-d065-4283-9367-c5e95fd8ff70 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:09.203590 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb2ab0c-11d1-44bd-ab24-f8f4fa77f6d6 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:09.234533 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a60d1-0d8d-4d1c-a745-a30a7dd7e8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:09.235197 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a60d1-0d8d-4d1c-a745-a30a7dd7e8fe None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:09.235476 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a60d1-0d8d-4d1c-a745-a30a7dd7e8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:09.235865 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a60d1-0d8d-4d1c-a745-a30a7dd7e8fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:09.240836 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85c949-9ef3-4da7-937c-6ba67838c06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:09.241437 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85c949-9ef3-4da7-937c-6ba67838c06e None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:09.241714 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85c949-9ef3-4da7-937c-6ba67838c06e None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:09.241985 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de85c949-9ef3-4da7-937c-6ba67838c06e None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:09.242620 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85c949-9ef3-4da7-937c-6ba67838c06e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:09.242917 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85c949-9ef3-4da7-937c-6ba67838c06e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:09.244751 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de85c949-9ef3-4da7-937c-6ba67838c06e None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:09.247147 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a60d1-0d8d-4d1c-a745-a30a7dd7e8fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:09.248691 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a60d1-0d8d-4d1c-a745-a30a7dd7e8fe None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:09.262960 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 243/470] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:09.262960 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85c949-9ef3-4da7-937c-6ba67838c06e None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:09.269027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-822a60d1-0d8d-4d1c-a745-a30a7dd7e8fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:09.269027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:09.269027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:09.269027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:09.269027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:09.269027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:09.269027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:09.269027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:09.269027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:09.269027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:09.269027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:09.269027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:09.269027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:09.269027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:09.269027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:09.269027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:09.270646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:09.270646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:09.270646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:09.270646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:09.270646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:09.270646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:09.270646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:09.270646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:09.270646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:09.270646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:09.270646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:09.270646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:09.270646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:09.270646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:09.270646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:09.270646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:09.270646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:09.276215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:09.276215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:09.276215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:09.276215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 228/471] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:09.276215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c2cc2-55fa-41a3-bfbb-41cc049becb7 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:09.276215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c2cc2-55fa-41a3-bfbb-41cc049becb7 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:09.276215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c2cc2-55fa-41a3-bfbb-41cc049becb7 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:09.276215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 229/472] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:09.276215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c2cc2-55fa-41a3-bfbb-41cc049becb7 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:09.337438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:09.381973 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de85c949-9ef3-4da7-937c-6ba67838c06e None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:09.386264 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c2cc2-55fa-41a3-bfbb-41cc049becb7 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:09.399563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:09.419687 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b6295-1877-471c-b377-e56f3cf8dab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:09.420302 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b6295-1877-471c-b377-e56f3cf8dab8 None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:09.420563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b6295-1877-471c-b377-e56f3cf8dab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:09.420832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b6295-1877-471c-b377-e56f3cf8dab8 None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:09.421465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b6295-1877-471c-b377-e56f3cf8dab8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:09.422463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e0ff1-3db9-4da7-b152-c91b28e89978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:09.422895 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b6295-1877-471c-b377-e56f3cf8dab8 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:09.423061 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e0ff1-3db9-4da7-b152-c91b28e89978 None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:09.423470 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e0ff1-3db9-4da7-b152-c91b28e89978 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:09.423746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e0ff1-3db9-4da7-b152-c91b28e89978 None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:09.424359 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e0ff1-3db9-4da7-b152-c91b28e89978 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:09.425781 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e0ff1-3db9-4da7-b152-c91b28e89978 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:09.447009 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 244/473] 10.222.0.36 () {60 vars in 1132 bytes} [Fri Sep 2 08:56:09 2022] POST /identity/v3/users => generated 248 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:56:09.447840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b6295-1877-471c-b377-e56f3cf8dab8 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:09.448493 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 230/474] 10.222.0.36 () {60 vars in 1132 bytes} [Fri Sep 2 08:56:09 2022] POST /identity/v3/users => generated 248 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:56:09.449903 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ee135-0fba-4927-afb5-0ec8275bda5b None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:09.450402 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ee135-0fba-4927-afb5-0ec8275bda5b None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:09.450874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ee135-0fba-4927-afb5-0ec8275bda5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:09.496089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-290ee135-0fba-4927-afb5-0ec8275bda5b None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:09.498926 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-290ee135-0fba-4927-afb5-0ec8275bda5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:09.630392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b6295-1877-471c-b377-e56f3cf8dab8 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:09.633702 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 231/475] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:09.669878 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655a747-2801-4cfe-ab61-7cb3fd910962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:09.670897 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655a747-2801-4cfe-ab61-7cb3fd910962 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:09.671393 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655a747-2801-4cfe-ab61-7cb3fd910962 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:09.671867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655a747-2801-4cfe-ab61-7cb3fd910962 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:09.672512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6655a747-2801-4cfe-ab61-7cb3fd910962 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:09.672811 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6655a747-2801-4cfe-ab61-7cb3fd910962 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:09.674215 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6655a747-2801-4cfe-ab61-7cb3fd910962 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:09.687823 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 245/476] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:09.695255 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ee135-0fba-4927-afb5-0ec8275bda5b None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:09.741433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:09.806538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ee135-0fba-4927-afb5-0ec8275bda5b None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:09.843394 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0d7f6-ca8f-48a0-ba70-18839bbd42b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:09.843998 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0d7f6-ca8f-48a0-ba70-18839bbd42b9 None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:09.844274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0d7f6-ca8f-48a0-ba70-18839bbd42b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:09.844540 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0d7f6-ca8f-48a0-ba70-18839bbd42b9 None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:09.845188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0d7f6-ca8f-48a0-ba70-18839bbd42b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:09.846671 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0d7f6-ca8f-48a0-ba70-18839bbd42b9 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:09.871012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 232/477] 10.222.0.36 () {60 vars in 1132 bytes} [Fri Sep 2 08:56:09 2022] POST /identity/v3/users => generated 249 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:56:09.912510 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655a747-2801-4cfe-ab61-7cb3fd910962 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:09.949621 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:10.040412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655a747-2801-4cfe-ab61-7cb3fd910962 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:10.083164 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8716736-6cc0-48df-a712-3a36299684d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:10.084054 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8716736-6cc0-48df-a712-3a36299684d9 None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:10.084477 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8716736-6cc0-48df-a712-3a36299684d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:10.084887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8716736-6cc0-48df-a712-3a36299684d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:10.085700 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8716736-6cc0-48df-a712-3a36299684d9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:10.087255 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8716736-6cc0-48df-a712-3a36299684d9 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:10.100217 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.sql.core [None req-a8716736-6cc0-48df-a712-3a36299684d9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 02 08:56:10.100217 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 02 08:56:10.100217 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [parameters: {'id': 'a69db9004702413397aaac006aa05c28', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 02 08:56:10.100217 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81389) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 02 08:56:10.116512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-a8716736-6cc0-48df-a712-3a36299684d9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 02 08:56:10.116512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:10.116512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 02 08:56:10.116512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 02 08:56:10.116512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 02 08:56:10.116512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 02 08:56:10.116512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 02 08:56:10.116512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application result = self._query(query) Sep 02 08:56:10.116512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 02 08:56:10.116512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application conn.query(q) Sep 02 08:56:10.116512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 02 08:56:10.116512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 02 08:56:10.116512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 02 08:56:10.116512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application result.read() Sep 02 08:56:10.116512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 02 08:56:10.117594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 02 08:56:10.117594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 02 08:56:10.117594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 02 08:56:10.117594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 02 08:56:10.117594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 02 08:56:10.117594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 02 08:56:10.117594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 02 08:56:10.117594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 02 08:56:10.117594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:10.117594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 02 08:56:10.117594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:10.117594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:10.117594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 02 08:56:10.117594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 02 08:56:10.117594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 02 08:56:10.117594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return ref.to_dict() Sep 02 08:56:10.117594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 02 08:56:10.117594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application next(self.gen) Sep 02 08:56:10.118706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 02 08:56:10.118706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application yield resource Sep 02 08:56:10.118706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 02 08:56:10.118706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application next(self.gen) Sep 02 08:56:10.118706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 02 08:56:10.118706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self.session.rollback() Sep 02 08:56:10.118706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 02 08:56:10.118706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self.force_reraise() Sep 02 08:56:10.118706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 02 08:56:10.118706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise self.value Sep 02 08:56:10.118706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 02 08:56:10.118706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 02 08:56:10.118706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 02 08:56:10.118706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application session.commit() Sep 02 08:56:10.118706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 02 08:56:10.118706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 02 08:56:10.118706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 02 08:56:10.119829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._prepare_impl() Sep 02 08:56:10.119829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 02 08:56:10.119829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self.session.flush() Sep 02 08:56:10.119829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 02 08:56:10.119829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._flush(objects) Sep 02 08:56:10.119829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 02 08:56:10.119829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 02 08:56:10.119829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 02 08:56:10.119829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application compat.raise_( Sep 02 08:56:10.119829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 02 08:56:10.119829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception Sep 02 08:56:10.119829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 02 08:56:10.119829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application flush_context.execute() Sep 02 08:56:10.119829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 02 08:56:10.119829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rec.execute(self) Sep 02 08:56:10.119829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 02 08:56:10.119829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 02 08:56:10.120921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 02 08:56:10.120921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 02 08:56:10.120921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 02 08:56:10.120921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 02 08:56:10.120921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 02 08:56:10.120921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 02 08:56:10.120921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 02 08:56:10.120921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 02 08:56:10.120921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 02 08:56:10.120921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 02 08:56:10.120921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 02 08:56:10.120921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 02 08:56:10.120921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 02 08:56:10.120921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 02 08:56:10.120921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 02 08:56:10.120921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception Sep 02 08:56:10.121927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 02 08:56:10.121927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 02 08:56:10.121927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 02 08:56:10.121927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 02 08:56:10.121927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 02 08:56:10.121927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application result = self._query(query) Sep 02 08:56:10.121927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 02 08:56:10.121927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application conn.query(q) Sep 02 08:56:10.121927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 02 08:56:10.121927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 02 08:56:10.121927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 02 08:56:10.121927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application result.read() Sep 02 08:56:10.121927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 02 08:56:10.121927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 02 08:56:10.121927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 02 08:56:10.121927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 02 08:56:10.123711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 02 08:56:10.123711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 02 08:56:10.123711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 02 08:56:10.123711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 02 08:56:10.123711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 02 08:56:10.123711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 02 08:56:10.123711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [parameters: {'id': 'a69db9004702413397aaac006aa05c28', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 02 08:56:10.123711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 02 08:56:10.123711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:10.123711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:56:10.123711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:10.123711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:10.123711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:10.123711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:10.123711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:10.123711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:10.125709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:10.125709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:10.125709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:10.125709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:10.125709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:10.125709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:10.125709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 02 08:56:10.125709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 02 08:56:10.125709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:10.125709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:10.125709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 02 08:56:10.125709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 02 08:56:10.125709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 02 08:56:10.125709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 02 08:56:10.125709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 02 08:56:10.125709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:10.129770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 246/478] 10.222.0.36 () {60 vars in 1131 bytes} [Fri Sep 2 08:56:09 2022] POST /identity/v3/roles => generated 144 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:10.129906 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0d7f6-ca8f-48a0-ba70-18839bbd42b9 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:10.236945 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:10.246824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0d7f6-ca8f-48a0-ba70-18839bbd42b9 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:10.288057 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443aca49-c4b9-429c-b7bd-d0d7e9504c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:10.289041 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443aca49-c4b9-429c-b7bd-d0d7e9504c4e None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:10.289558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443aca49-c4b9-429c-b7bd-d0d7e9504c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:10.290020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443aca49-c4b9-429c-b7bd-d0d7e9504c4e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:10.300986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443aca49-c4b9-429c-b7bd-d0d7e9504c4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:10.306627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-443aca49-c4b9-429c-b7bd-d0d7e9504c4e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:56:10.306627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:10.306627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:10.306627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:10.306627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:10.306627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:10.306627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:10.306627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:10.306627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:10.306627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:10.306627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:10.306627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:10.306627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:10.306627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:10.306627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:10.306627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:10.307564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:10.307564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:10.307564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:10.307564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:10.307564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:10.307564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:10.307564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:10.307564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:10.307564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:10.307564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:10.307564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:10.307564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:10.307564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:10.307564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:10.307564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:10.307564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:10.308346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:10.308346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:10.308346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:10.308346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:10.308346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:10.308346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:10.308346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:10.308346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:10.308346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:10.308346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:10.308346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 08:56:10.308346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:10.315165 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 233/479] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:56:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:10.317158 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8716736-6cc0-48df-a712-3a36299684d9 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:10.443515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:10.447628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8716736-6cc0-48df-a712-3a36299684d9 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:10.495000 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f697eb4-3ea2-459c-9cb2-0f7dbce7f139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:10.495559 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f697eb4-3ea2-459c-9cb2-0f7dbce7f139 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:10.495843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f697eb4-3ea2-459c-9cb2-0f7dbce7f139 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:10.496166 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f697eb4-3ea2-459c-9cb2-0f7dbce7f139 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:10.496840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f697eb4-3ea2-459c-9cb2-0f7dbce7f139 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:10.497145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f697eb4-3ea2-459c-9cb2-0f7dbce7f139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:10.498573 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f697eb4-3ea2-459c-9cb2-0f7dbce7f139 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:10.507798 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 247/480] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:10.953074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddf29b-aa21-4ed2-bb25-fdafc9f3aae4 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:10.953631 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddf29b-aa21-4ed2-bb25-fdafc9f3aae4 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:10.953631 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caddf29b-aa21-4ed2-bb25-fdafc9f3aae4 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:10.955256 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 234/481] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:10.960687 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a1fce-2ae0-4ecc-8d29-c2c876ce23ed None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:10.961117 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a1fce-2ae0-4ecc-8d29-c2c876ce23ed None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:10.961309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a1fce-2ae0-4ecc-8d29-c2c876ce23ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:10.979773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-9b4a1fce-2ae0-4ecc-8d29-c2c876ce23ed None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:10.980960 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-9b4a1fce-2ae0-4ecc-8d29-c2c876ce23ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:11.055985 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 248/482] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:11.413726 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caddf29b-aa21-4ed2-bb25-fdafc9f3aae4 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:11.549265 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caddf29b-aa21-4ed2-bb25-fdafc9f3aae4 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:11.593665 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c22523-5068-4986-83ed-91f31d62839e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:11.594298 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c22523-5068-4986-83ed-91f31d62839e None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:11.594580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c22523-5068-4986-83ed-91f31d62839e None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:11.594953 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c22523-5068-4986-83ed-91f31d62839e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:11.612079 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c22523-5068-4986-83ed-91f31d62839e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:11.617466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-57c22523-5068-4986-83ed-91f31d62839e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:11.617466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:11.617466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:11.617466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:11.617466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:11.617466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:11.617466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:11.617466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:11.617466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:11.617466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:11.617466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:11.617466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:11.617466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:11.617466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:11.617466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:11.617466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:11.618355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:11.618355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:11.618355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:11.618355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:11.618355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:11.618355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:11.618355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:11.618355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:11.618355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:11.618355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:11.618355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:11.618355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:11.618355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:11.618355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:11.618355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:11.618355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:11.619191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:11.619191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:11.619191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:11.619191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:11.619191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:11.619191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:11.619191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:11.619191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:11.619191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:11.619191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:11.619191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:11.619191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:11.619191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 235/483] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:56:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:11.624318 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a1fce-2ae0-4ecc-8d29-c2c876ce23ed None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:11.719503 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:11.737935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a1fce-2ae0-4ecc-8d29-c2c876ce23ed None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:11.789545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3f0cb-9ca8-44f8-a83b-d6f13b26a7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:11.790160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3f0cb-9ca8-44f8-a83b-d6f13b26a7b5 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:11.790441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3f0cb-9ca8-44f8-a83b-d6f13b26a7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:11.790717 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a3f0cb-9ca8-44f8-a83b-d6f13b26a7b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:11.791412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a3f0cb-9ca8-44f8-a83b-d6f13b26a7b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:11.791719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a3f0cb-9ca8-44f8-a83b-d6f13b26a7b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:11.793099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a3f0cb-9ca8-44f8-a83b-d6f13b26a7b5 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:11.802425 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 249/484] 10.222.0.36 () {56 vars in 1103 bytes} [Fri Sep 2 08:56:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:11.807614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c22523-5068-4986-83ed-91f31d62839e None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:11.833747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:11.932275 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c22523-5068-4986-83ed-91f31d62839e None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:11.981751 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2fa4f-9634-48a9-b243-cfdc157eace4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:11.982378 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2fa4f-9634-48a9-b243-cfdc157eace4 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:11.982660 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2fa4f-9634-48a9-b243-cfdc157eace4 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:11.982934 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2fa4f-9634-48a9-b243-cfdc157eace4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:11.998418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2fa4f-9634-48a9-b243-cfdc157eace4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:11.999974 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2fa4f-9634-48a9-b243-cfdc157eace4 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:12.022900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-5fb2fa4f-9634-48a9-b243-cfdc157eace4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:12.022900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:12.022900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:12.022900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:12.022900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:12.022900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:12.022900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:12.022900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:12.022900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:12.022900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:12.022900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:12.022900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:12.022900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:12.022900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:12.022900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:12.022900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:12.023873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:12.023873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:12.023873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:12.023873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:12.023873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:12.023873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:12.023873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:12.023873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:12.023873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:12.023873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:12.023873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:12.023873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:12.023873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:12.023873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:12.023873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:12.023873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:12.023873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:12.025093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:12.025093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:12.025093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:12.025093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 236/485] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:12.029753 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3f0cb-9ca8-44f8-a83b-d6f13b26a7b5 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:12.061416 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed2ee6-3262-412c-ab83-5c3852b35c07 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:12.061754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed2ee6-3262-412c-ab83-5c3852b35c07 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:12.062029 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed2ee6-3262-412c-ab83-5c3852b35c07 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:12.063401 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 237/486] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:56:12.068594 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c153a-5c44-4799-bc7c-fc32ac707c05 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:12.068881 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c153a-5c44-4799-bc7c-fc32ac707c05 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:12.069158 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c153a-5c44-4799-bc7c-fc32ac707c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:12.091283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-ae8c153a-5c44-4799-bc7c-fc32ac707c05 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:12.098913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ae8c153a-5c44-4799-bc7c-fc32ac707c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:12.211523 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:12.216929 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a3f0cb-9ca8-44f8-a83b-d6f13b26a7b5 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:12.221478 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 238/487] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:56:12.251455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37bfa7-2611-4ae6-b606-2cf5e484dc7f None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:12.251764 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37bfa7-2611-4ae6-b606-2cf5e484dc7f None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:12.252070 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f37bfa7-2611-4ae6-b606-2cf5e484dc7f None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:12.253407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 239/488] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:12.258450 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4d51a-d968-463a-a9e9-9c73cf0f3164 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:12.258732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4d51a-d968-463a-a9e9-9c73cf0f3164 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:12.259006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4d51a-d968-463a-a9e9-9c73cf0f3164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:12.266072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d8c15-f70b-46eb-b393-aad61db4d4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:12.266682 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d8c15-f70b-46eb-b393-aad61db4d4de None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:12.266964 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d8c15-f70b-46eb-b393-aad61db4d4de None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:12.267266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d8c15-f70b-46eb-b393-aad61db4d4de None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:12.268949 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d8c15-f70b-46eb-b393-aad61db4d4de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:12.269257 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d8c15-f70b-46eb-b393-aad61db4d4de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:12.272529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d8c15-f70b-46eb-b393-aad61db4d4de None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:12.285323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-faa4d51a-d968-463a-a9e9-9c73cf0f3164 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:12.287822 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-faa4d51a-d968-463a-a9e9-9c73cf0f3164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:12.302472 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 250/489] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:12.310007 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d8c15-f70b-46eb-b393-aad61db4d4de None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:12.336354 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:12.423538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 240/490] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:56:12.454676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d8c15-f70b-46eb-b393-aad61db4d4de None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:12.454999 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca43ed-c41d-40b6-9d31-dc1380aff229 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:12.455338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca43ed-c41d-40b6-9d31-dc1380aff229 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:12.455617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca43ed-c41d-40b6-9d31-dc1380aff229 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:12.458023 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 241/491] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:56:12.463036 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d72cf9-e6da-4c24-a498-3ebb3d693d97 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:12.463287 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d72cf9-e6da-4c24-a498-3ebb3d693d97 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:12.463569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d72cf9-e6da-4c24-a498-3ebb3d693d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:12.496493 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2e2d9-8ad4-4a51-b1d7-d9ae607c2f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:12.497116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2e2d9-8ad4-4a51-b1d7-d9ae607c2f55 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:12.497395 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2e2d9-8ad4-4a51-b1d7-d9ae607c2f55 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:12.497666 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd2e2d9-8ad4-4a51-b1d7-d9ae607c2f55 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:12.503425 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-b7d72cf9-e6da-4c24-a498-3ebb3d693d97 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:12.506575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b7d72cf9-e6da-4c24-a498-3ebb3d693d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:12.515004 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2e2d9-8ad4-4a51-b1d7-d9ae607c2f55 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:12.519578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd2e2d9-8ad4-4a51-b1d7-d9ae607c2f55 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:12.540397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-7dd2e2d9-8ad4-4a51-b1d7-d9ae607c2f55 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 08:56:12.540397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:12.540397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:12.540397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:12.540397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:12.540397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:12.540397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:12.540397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:12.540397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:12.540397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:12.540397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:12.540397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:12.540397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:12.540397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:12.540397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:12.540397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:12.541321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:12.541321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:12.541321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:12.541321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:12.541321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:12.541321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:12.541321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:12.541321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:12.541321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:12.541321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:12.541321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:12.541321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:12.541321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:12.541321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:12.541321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:12.541321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:12.541321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:12.541321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:12.542627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:12.542627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:12.542627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:12.542627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:12.542627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:12.542627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:12.542627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:12.542627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:12.542627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 08:56:12.542627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:12.542627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 251/492] 10.222.0.36 () {56 vars in 1093 bytes} [Fri Sep 2 08:56:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:12.548690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2e2d9-8ad4-4a51-b1d7-d9ae607c2f55 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:12.602302 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 242/493] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:56:12.603722 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d72cf9-e6da-4c24-a498-3ebb3d693d97 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:12.624974 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:12.657301 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd2e2d9-8ad4-4a51-b1d7-d9ae607c2f55 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:12.668585 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:12.693544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08067a13-f986-42dd-aa4f-cc1da95bae10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:12.694155 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08067a13-f986-42dd-aa4f-cc1da95bae10 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:12.694436 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08067a13-f986-42dd-aa4f-cc1da95bae10 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:12.694882 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08067a13-f986-42dd-aa4f-cc1da95bae10 None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:12.695647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08067a13-f986-42dd-aa4f-cc1da95bae10 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:12.695951 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08067a13-f986-42dd-aa4f-cc1da95bae10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:12.697452 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08067a13-f986-42dd-aa4f-cc1da95bae10 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:12.715642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d72cf9-e6da-4c24-a498-3ebb3d693d97 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:12.735673 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 252/494] 10.222.0.36 () {56 vars in 1133 bytes} [Fri Sep 2 08:56:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:12.736530 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08067a13-f986-42dd-aa4f-cc1da95bae10 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:12.753897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3038e0-9a66-4aa9-9056-257e89ea22fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:12.754876 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3038e0-9a66-4aa9-9056-257e89ea22fb None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:12.755745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3038e0-9a66-4aa9-9056-257e89ea22fb None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:12.756244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3038e0-9a66-4aa9-9056-257e89ea22fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:12.767970 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3038e0-9a66-4aa9-9056-257e89ea22fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:12.773569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-0a3038e0-9a66-4aa9-9056-257e89ea22fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:12.773569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:12.773569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:12.773569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:12.773569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:12.773569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:12.773569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:12.773569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:12.773569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:12.773569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:12.773569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:12.773569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:12.773569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:12.773569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:12.773569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:12.773569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:12.774480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:12.774480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:12.774480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:12.774480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:12.774480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:12.774480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:12.774480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:12.774480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:12.774480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:12.774480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:12.774480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:12.774480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:12.774480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:12.774480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:12.774480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:12.774480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:12.775282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:12.775282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:12.775282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:12.775282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:12.775282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:12.775282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:12.775282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:12.775282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:12.775282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:12.775282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:12.775282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:12.775282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:12.777578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 243/495] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:56:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:12.778670 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3038e0-9a66-4aa9-9056-257e89ea22fb None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:12.842070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08067a13-f986-42dd-aa4f-cc1da95bae10 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:12.873385 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:12.879425 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6aae15-e2c8-4851-9bfc-8b90efb69cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:12.880085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6aae15-e2c8-4851-9bfc-8b90efb69cbc None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:12.880371 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6aae15-e2c8-4851-9bfc-8b90efb69cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:12.880699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6aae15-e2c8-4851-9bfc-8b90efb69cbc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:12.891161 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3038e0-9a66-4aa9-9056-257e89ea22fb None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:12.893275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6aae15-e2c8-4851-9bfc-8b90efb69cbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:12.898315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-5e6aae15-e2c8-4851-9bfc-8b90efb69cbc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:12.898315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:12.898315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:12.898315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:12.898315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:12.898315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:12.898315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:12.898315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:12.898315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:12.898315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:12.898315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:12.898315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:12.898315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:12.898315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:12.898315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:12.898315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:12.899238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:12.899238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:12.899238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:12.899238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:12.899238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:12.899238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:12.899238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:12.899238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:12.899238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:12.899238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:12.899238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:12.899238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:12.899238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:12.899238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:12.899238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:12.899238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:12.900035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:12.900035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:12.900035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:12.900035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:12.900035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:12.900035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:12.900035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:12.900035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:12.900035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:12.900035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:12.900035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:12.900035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:12.902291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 253/496] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:56:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:12.903390 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6aae15-e2c8-4851-9bfc-8b90efb69cbc None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:12.915019 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:12.934905 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e120f-17b9-458a-b9cd-c4bca0f95f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:12.935753 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e120f-17b9-458a-b9cd-c4bca0f95f90 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:12.936055 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e120f-17b9-458a-b9cd-c4bca0f95f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:12.936331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e120f-17b9-458a-b9cd-c4bca0f95f90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:13.005506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e120f-17b9-458a-b9cd-c4bca0f95f90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:13.007471 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e120f-17b9-458a-b9cd-c4bca0f95f90 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:13.018577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6aae15-e2c8-4851-9bfc-8b90efb69cbc None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:13.056811 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6099ab8-ee8b-49be-9472-1798455820d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:13.057547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6099ab8-ee8b-49be-9472-1798455820d9 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:13.057735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6099ab8-ee8b-49be-9472-1798455820d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:13.058061 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6099ab8-ee8b-49be-9472-1798455820d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:13.058806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6099ab8-ee8b-49be-9472-1798455820d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:13.059138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6099ab8-ee8b-49be-9472-1798455820d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:13.060787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6099ab8-ee8b-49be-9472-1798455820d9 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:13.074019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 254/497] 10.222.0.36 () {56 vars in 1103 bytes} [Fri Sep 2 08:56:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:13.074019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6099ab8-ee8b-49be-9472-1798455820d9 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:13.114842 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 244/498] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:56:13.114842 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e120f-17b9-458a-b9cd-c4bca0f95f90 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:13.129262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:13.135892 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:13.183119 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6099ab8-ee8b-49be-9472-1798455820d9 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:13.219309 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e120f-17b9-458a-b9cd-c4bca0f95f90 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:13.219515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a2c2f-7a34-461d-8079-f98b7937f7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:13.220127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a2c2f-7a34-461d-8079-f98b7937f7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:13.220408 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a2c2f-7a34-461d-8079-f98b7937f7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:13.220681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84a2c2f-7a34-461d-8079-f98b7937f7e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:13.221337 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a2c2f-7a34-461d-8079-f98b7937f7e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:13.221633 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a2c2f-7a34-461d-8079-f98b7937f7e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:13.223648 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84a2c2f-7a34-461d-8079-f98b7937f7e0 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:13.236174 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 255/499] 10.222.0.36 () {56 vars in 1103 bytes} [Fri Sep 2 08:56:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:13.236976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a2c2f-7a34-461d-8079-f98b7937f7e0 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:13.255864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb4bb2-e947-419a-9819-2f52e26f4cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:13.256731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb4bb2-e947-419a-9819-2f52e26f4cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:13.257249 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb4bb2-e947-419a-9819-2f52e26f4cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:13.257691 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb4bb2-e947-419a-9819-2f52e26f4cd5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:13.268547 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb4bb2-e947-419a-9819-2f52e26f4cd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:13.274400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-3bfb4bb2-e947-419a-9819-2f52e26f4cd5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:13.274400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:13.274400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:13.274400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:13.274400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:13.274400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:13.274400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:13.274400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:13.274400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:13.274400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:13.274400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:13.274400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:13.274400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:13.274400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:13.274400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:13.274400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:13.275418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:13.275418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:13.275418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:13.275418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:13.275418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:13.275418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:13.275418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:13.275418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:13.275418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:13.275418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:13.275418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:13.275418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:13.275418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:13.275418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:13.275418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:13.275418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:13.276247 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:13.276247 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:13.276247 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:13.276247 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:13.276247 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:13.276247 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:13.276247 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:13.276247 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:13.276247 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:13.276247 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:13.276247 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:13.276247 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:13.276852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 245/500] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:56:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:13.277343 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb4bb2-e947-419a-9819-2f52e26f4cd5 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:13.343468 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84a2c2f-7a34-461d-8079-f98b7937f7e0 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:13.357970 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:13.362804 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:13.379124 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be413631-8db9-4567-a833-458553d9f517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:13.379757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be413631-8db9-4567-a833-458553d9f517 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:13.380041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be413631-8db9-4567-a833-458553d9f517 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:13.380314 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be413631-8db9-4567-a833-458553d9f517 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:13.382042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb4bb2-e947-419a-9819-2f52e26f4cd5 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:13.390554 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be413631-8db9-4567-a833-458553d9f517 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:13.392031 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be413631-8db9-4567-a833-458553d9f517 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:13.406349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-be413631-8db9-4567-a833-458553d9f517 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:13.406349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:13.406349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:13.406349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:13.406349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:13.406349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:13.406349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:13.406349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:13.406349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:13.406349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:13.406349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:13.406349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:13.406349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:13.406349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:13.406349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:13.406349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:13.407329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:13.407329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:13.407329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:13.407329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:13.407329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:13.407329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:13.407329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:13.407329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:13.407329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:13.407329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:13.407329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:13.407329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:13.407329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:13.407329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:13.407329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:13.407329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:13.407329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:13.408584 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:13.408584 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:13.408584 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:13.408584 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 256/501] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:13.408584 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be413631-8db9-4567-a833-458553d9f517 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:13.418804 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb874e00-1461-42ef-91c6-d6e7ac5267b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:13.419469 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb874e00-1461-42ef-91c6-d6e7ac5267b0 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:13.419752 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb874e00-1461-42ef-91c6-d6e7ac5267b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:13.420026 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb874e00-1461-42ef-91c6-d6e7ac5267b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:13.430401 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb874e00-1461-42ef-91c6-d6e7ac5267b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:13.431874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb874e00-1461-42ef-91c6-d6e7ac5267b0 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:13.446712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-eb874e00-1461-42ef-91c6-d6e7ac5267b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:13.446712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:13.446712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:13.446712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:13.446712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:13.446712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:13.446712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:13.446712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:13.446712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:13.446712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:13.446712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:13.446712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:13.446712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:13.446712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:13.446712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:13.446712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:13.447633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:13.447633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:13.447633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:13.447633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:13.447633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:13.447633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:13.447633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:13.447633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:13.447633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:13.447633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:13.447633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:13.447633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:13.447633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:13.447633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:13.447633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:13.447633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:13.447633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:13.448490 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:13.448490 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:13.448490 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:13.448490 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 246/502] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:13.448712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb874e00-1461-42ef-91c6-d6e7ac5267b0 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:13.511911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be413631-8db9-4567-a833-458553d9f517 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:13.546215 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87feb50f-7cb1-460a-bae2-f47921289301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:13.546377 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb874e00-1461-42ef-91c6-d6e7ac5267b0 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:13.546894 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87feb50f-7cb1-460a-bae2-f47921289301 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:13.547210 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87feb50f-7cb1-460a-bae2-f47921289301 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:13.547490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87feb50f-7cb1-460a-bae2-f47921289301 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:13.558137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87feb50f-7cb1-460a-bae2-f47921289301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:13.559859 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87feb50f-7cb1-460a-bae2-f47921289301 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:13.574781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-87feb50f-7cb1-460a-bae2-f47921289301 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:13.574781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:13.574781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:13.574781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:13.574781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:13.574781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:13.574781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:13.574781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:13.574781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:13.574781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:13.574781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:13.574781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:13.574781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:13.574781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:13.574781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:13.574781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:13.580727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:13.580727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:13.580727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:13.580727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:13.580727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:13.580727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:13.580727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:13.580727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:13.580727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:13.580727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:13.580727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:13.580727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:13.580727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:13.580727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:13.580727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:13.580727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:13.580727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:13.581690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:13.581690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:13.581690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:13.581856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:13.581926 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 257/503] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:13.581926 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89496981-cbde-4810-bd4a-e7c6683b4eaa None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:13.581926 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89496981-cbde-4810-bd4a-e7c6683b4eaa None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:13.581926 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89496981-cbde-4810-bd4a-e7c6683b4eaa None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:13.581926 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 258/504] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:13.581926 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89496981-cbde-4810-bd4a-e7c6683b4eaa None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:13.586275 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e49349-29f7-4972-84bb-e051f1d89287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:13.586884 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e49349-29f7-4972-84bb-e051f1d89287 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:13.587203 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e49349-29f7-4972-84bb-e051f1d89287 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:13.587477 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e49349-29f7-4972-84bb-e051f1d89287 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:13.588144 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e49349-29f7-4972-84bb-e051f1d89287 None admin] RBAC: Adding query filter params (name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:13.588449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e49349-29f7-4972-84bb-e051f1d89287 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:13.589812 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e49349-29f7-4972-84bb-e051f1d89287 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:13.598126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 247/505] 10.222.0.36 () {56 vars in 1103 bytes} [Fri Sep 2 08:56:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:13.598895 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e49349-29f7-4972-84bb-e051f1d89287 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:13.599985 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:13.682678 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89496981-cbde-4810-bd4a-e7c6683b4eaa None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:13.696132 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e49349-29f7-4972-84bb-e051f1d89287 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:13.717509 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e7987-b842-4be0-a1ab-e55c34c4087f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:13.718334 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e7987-b842-4be0-a1ab-e55c34c4087f None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:13.718612 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e7987-b842-4be0-a1ab-e55c34c4087f None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:13.718888 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e7987-b842-4be0-a1ab-e55c34c4087f None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:13.719553 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53e7987-b842-4be0-a1ab-e55c34c4087f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:13.719852 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53e7987-b842-4be0-a1ab-e55c34c4087f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:13.721233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53e7987-b842-4be0-a1ab-e55c34c4087f None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:13.729580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dc02b9-310d-4b06-aaf6-f3f942941c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:13.730176 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc02b9-310d-4b06-aaf6-f3f942941c6c None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:13.730456 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc02b9-310d-4b06-aaf6-f3f942941c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:13.730719 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dc02b9-310d-4b06-aaf6-f3f942941c6c None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:13.731378 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc02b9-310d-4b06-aaf6-f3f942941c6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:13.731678 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc02b9-310d-4b06-aaf6-f3f942941c6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:13.733019 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dc02b9-310d-4b06-aaf6-f3f942941c6c None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:13.738017 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 259/506] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:13.738017 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e7987-b842-4be0-a1ab-e55c34c4087f None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:13.748324 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 248/507] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:13.749486 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c41358-8f3d-4d10-92b9-93c94e362e91 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:13.749768 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c41358-8f3d-4d10-92b9-93c94e362e91 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:13.750036 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c41358-8f3d-4d10-92b9-93c94e362e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:13.767578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-51c41358-8f3d-4d10-92b9-93c94e362e91 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:13.768700 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-51c41358-8f3d-4d10-92b9-93c94e362e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:13.814420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:13.820962 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:13.836766 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e7987-b842-4be0-a1ab-e55c34c4087f None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:13.838719 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 249/508] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:13.849476 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c41358-8f3d-4d10-92b9-93c94e362e91 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:13.882255 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860737ae-2374-4cac-ae47-04bac7a2de95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:13.882957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860737ae-2374-4cac-ae47-04bac7a2de95 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:13.883383 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860737ae-2374-4cac-ae47-04bac7a2de95 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:13.883727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860737ae-2374-4cac-ae47-04bac7a2de95 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:13.884430 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860737ae-2374-4cac-ae47-04bac7a2de95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:13.884796 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860737ae-2374-4cac-ae47-04bac7a2de95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:13.886229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860737ae-2374-4cac-ae47-04bac7a2de95 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:13.898117 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 260/509] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:13.898871 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860737ae-2374-4cac-ae47-04bac7a2de95 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:13.945968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c41358-8f3d-4d10-92b9-93c94e362e91 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:13.980788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb7e08-7e7f-41ab-be29-f8917472c714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:13.981408 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb7e08-7e7f-41ab-be29-f8917472c714 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:13.981682 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb7e08-7e7f-41ab-be29-f8917472c714 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:13.981949 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb7e08-7e7f-41ab-be29-f8917472c714 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:13.992113 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb7e08-7e7f-41ab-be29-f8917472c714 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:13.993544 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb7e08-7e7f-41ab-be29-f8917472c714 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:14.007788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-33fb7e08-7e7f-41ab-be29-f8917472c714 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:14.007788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:14.007788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:14.007788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:14.007788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:14.007788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:14.007788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:14.007788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:14.007788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:14.007788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:14.007788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:14.007788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:14.007788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:14.007788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:14.007788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:14.007788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:14.009111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:14.009111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:14.009111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:14.009111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:14.009111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:14.009111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:14.009111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:14.009111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:14.009111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:14.009111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:14.009111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:14.009111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:14.009111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:14.009111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:14.009111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:14.009111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:14.009111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:14.009949 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:14.009949 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:14.009949 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:14.009949 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 250/510] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:14.009949 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb7e08-7e7f-41ab-be29-f8917472c714 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:14.016836 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860737ae-2374-4cac-ae47-04bac7a2de95 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:14.046773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:14.055253 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779eaf1-f200-423c-8678-e18f1cfd0ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:14.055885 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779eaf1-f200-423c-8678-e18f1cfd0ebf None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:14.056168 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779eaf1-f200-423c-8678-e18f1cfd0ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:14.056443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779eaf1-f200-423c-8678-e18f1cfd0ebf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:14.064728 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779eaf1-f200-423c-8678-e18f1cfd0ebf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:14.066183 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779eaf1-f200-423c-8678-e18f1cfd0ebf None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:14.068091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:14.079412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-3779eaf1-f200-423c-8678-e18f1cfd0ebf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 08:56:14.079412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:14.079412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:14.079412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:14.079412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:14.079412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:14.079412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:14.079412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:14.079412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:14.079412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:14.079412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:14.079412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:14.079412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:14.079412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:14.079412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:14.079412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:14.080309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:14.080309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:14.080309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:14.080309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:14.080309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:14.080309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:14.080309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:14.080309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:14.080309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:14.080309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:14.080309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:14.080309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:14.080309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:14.080309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:14.080309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:14.080309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:14.080309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:14.080309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:14.081209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:14.081209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:14.081209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:14.081209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:14.081209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:14.081209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:14.081209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:14.081209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:14.081209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 08:56:14.081209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:14.081209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 261/511] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:56:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:14.082901 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779eaf1-f200-423c-8678-e18f1cfd0ebf None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:14.117785 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb7e08-7e7f-41ab-be29-f8917472c714 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:14.150356 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f720497-a76e-4402-aea8-20c5baab023b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:14.150982 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f720497-a76e-4402-aea8-20c5baab023b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:14.151296 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f720497-a76e-4402-aea8-20c5baab023b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:14.151575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f720497-a76e-4402-aea8-20c5baab023b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:14.158342 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f720497-a76e-4402-aea8-20c5baab023b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:14.159814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f720497-a76e-4402-aea8-20c5baab023b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:14.172020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-5f720497-a76e-4402-aea8-20c5baab023b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:14.172020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:14.172020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:14.172020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:14.172020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:14.172020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:14.172020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:14.172020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:14.172020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:14.172020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:14.172020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:14.172020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:14.172020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:14.172020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:14.172020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:14.172861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:14.172861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:14.172861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:14.172861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:14.172861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:14.172861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:14.172861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:14.172861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:14.172861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:14.172861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:14.172861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:14.172861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:14.172861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:14.172861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:14.172861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:14.172861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:14.172861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:14.174111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:14.174111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:14.174111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:14.174111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:14.174111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:14.174111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:14.174111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:14.174111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:14.174111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 251/512] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:14.174717 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f720497-a76e-4402-aea8-20c5baab023b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:14.185533 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779eaf1-f200-423c-8678-e18f1cfd0ebf None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:14.219604 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779765a-a999-4e6a-83e3-b84bf36a9e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:14.220221 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779765a-a999-4e6a-83e3-b84bf36a9e2a None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:14.220501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779765a-a999-4e6a-83e3-b84bf36a9e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:14.220772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779765a-a999-4e6a-83e3-b84bf36a9e2a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:14.228042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779765a-a999-4e6a-83e3-b84bf36a9e2a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:14.229484 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779765a-a999-4e6a-83e3-b84bf36a9e2a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:14.240382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-3779765a-a999-4e6a-83e3-b84bf36a9e2a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 02 08:56:14.240382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:14.240382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:14.240382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:14.240382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:14.240382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:14.240382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:14.240382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:14.240382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:14.240382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:14.240382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:14.240382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:14.240382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:14.240382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:14.240382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:14.240382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:14.241392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:14.241392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:14.241392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:14.241392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:14.241392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:14.241392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:14.241392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:14.241392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:14.241392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:14.241392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:14.241392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:14.241392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:14.241392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:14.241392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:14.241392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:14.241392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:14.241392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:14.241392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:14.242297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:14.242297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:14.242297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:14.242297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:14.242297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:14.242297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:14.242297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:14.242297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:14.242297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 02 08:56:14.242297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:14.242297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 262/513] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:56:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:14.242857 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779765a-a999-4e6a-83e3-b84bf36a9e2a None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:14.274397 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:14.281951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f720497-a76e-4402-aea8-20c5baab023b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:14.299037 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:14.314519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3370565-3c33-4c34-bf93-cabfb87e60c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:14.315143 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3370565-3c33-4c34-bf93-cabfb87e60c8 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:14.315425 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3370565-3c33-4c34-bf93-cabfb87e60c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:14.315697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3370565-3c33-4c34-bf93-cabfb87e60c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:14.316335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3370565-3c33-4c34-bf93-cabfb87e60c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:14.317162 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3370565-3c33-4c34-bf93-cabfb87e60c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:14.318529 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3370565-3c33-4c34-bf93-cabfb87e60c8 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:14.330842 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 252/514] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:14.331480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3370565-3c33-4c34-bf93-cabfb87e60c8 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:14.347687 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779765a-a999-4e6a-83e3-b84bf36a9e2a None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:14.380867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d17a0c-42ea-4132-8bfe-b04f25f15b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:14.381654 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d17a0c-42ea-4132-8bfe-b04f25f15b45 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:14.381928 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d17a0c-42ea-4132-8bfe-b04f25f15b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:14.382194 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d17a0c-42ea-4132-8bfe-b04f25f15b45 None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:14.382897 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d17a0c-42ea-4132-8bfe-b04f25f15b45 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:14.383229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d17a0c-42ea-4132-8bfe-b04f25f15b45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:14.384643 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d17a0c-42ea-4132-8bfe-b04f25f15b45 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:14.401448 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 263/515] 10.222.0.36 () {56 vars in 1137 bytes} [Fri Sep 2 08:56:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:14.402206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d17a0c-42ea-4132-8bfe-b04f25f15b45 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:14.430066 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3370565-3c33-4c34-bf93-cabfb87e60c8 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:14.463835 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b175697-7997-4efd-bbea-9c43348cd351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:14.464453 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b175697-7997-4efd-bbea-9c43348cd351 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:14.464729 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b175697-7997-4efd-bbea-9c43348cd351 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:14.464994 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b175697-7997-4efd-bbea-9c43348cd351 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:14.474701 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b175697-7997-4efd-bbea-9c43348cd351 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:14.476176 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b175697-7997-4efd-bbea-9c43348cd351 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:14.490259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-0b175697-7997-4efd-bbea-9c43348cd351 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:14.490259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:14.490259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:14.490259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:14.490259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:14.490259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:14.490259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:14.490259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:14.490259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:14.490259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:14.490259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:14.490259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:14.490259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:14.490259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:14.490259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:14.490259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:14.491232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:14.491232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:14.491232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:14.491232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:14.491232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:14.491232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:14.491232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:14.491232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:14.491232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:14.491232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:14.491232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:14.491232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:14.491232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:14.491232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:14.491232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:14.491232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:14.491232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:14.492086 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:14.492086 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:14.492086 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:14.492086 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 253/516] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:14.492086 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b175697-7997-4efd-bbea-9c43348cd351 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:14.499275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d17a0c-42ea-4132-8bfe-b04f25f15b45 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:14.505742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:14.516292 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:14.534401 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbed458-5e29-4694-a6ab-79e589aabb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:14.535019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbed458-5e29-4694-a6ab-79e589aabb64 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:14.535348 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbed458-5e29-4694-a6ab-79e589aabb64 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:14.535617 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbed458-5e29-4694-a6ab-79e589aabb64 None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:14.536292 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbed458-5e29-4694-a6ab-79e589aabb64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:14.536591 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbed458-5e29-4694-a6ab-79e589aabb64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:14.538007 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbed458-5e29-4694-a6ab-79e589aabb64 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:14.560497 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 264/517] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:14.560497 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbed458-5e29-4694-a6ab-79e589aabb64 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:14.598583 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b175697-7997-4efd-bbea-9c43348cd351 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:14.635503 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bdad04-8602-41fb-ae44-1a8171979137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:14.636110 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdad04-8602-41fb-ae44-1a8171979137 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:14.636384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdad04-8602-41fb-ae44-1a8171979137 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:14.636650 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdad04-8602-41fb-ae44-1a8171979137 None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:14.638658 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bdad04-8602-41fb-ae44-1a8171979137 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:14.638959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bdad04-8602-41fb-ae44-1a8171979137 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:14.640398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bdad04-8602-41fb-ae44-1a8171979137 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:14.667087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbed458-5e29-4694-a6ab-79e589aabb64 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:14.675738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 254/518] 10.222.0.36 () {56 vars in 1137 bytes} [Fri Sep 2 08:56:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:14.677046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bdad04-8602-41fb-ae44-1a8171979137 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:14.703990 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b31ac-e934-42de-8f96-241a0be0366a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:14.704614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b31ac-e934-42de-8f96-241a0be0366a None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:14.704890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b31ac-e934-42de-8f96-241a0be0366a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:14.705158 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b31ac-e934-42de-8f96-241a0be0366a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:14.712666 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b31ac-e934-42de-8f96-241a0be0366a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:14.714105 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b31ac-e934-42de-8f96-241a0be0366a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:14.725443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-c69b31ac-e934-42de-8f96-241a0be0366a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 02 08:56:14.725443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:14.725443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:14.725443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:14.725443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:14.725443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:14.725443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:14.725443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:14.725443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:14.725443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:14.725443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:14.725443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:14.725443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:14.725443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:14.725443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:14.725443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:14.731206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:14.731206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:14.731206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:14.731206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:14.731206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:14.731206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:14.731206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:14.731206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:14.731206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:14.731206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:14.731206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:14.731206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:14.731206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:14.731206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:14.731206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:14.731206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:14.731206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:14.731206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:14.732169 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:14.732169 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:14.732169 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:14.732169 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:14.732169 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:14.732169 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:14.732169 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:14.732169 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:14.732169 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 02 08:56:14.732169 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:14.732169 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 265/519] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:56:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:14.732169 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b31ac-e934-42de-8f96-241a0be0366a None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:14.742360 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:14.762917 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:14.778037 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bdad04-8602-41fb-ae44-1a8171979137 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:14.812496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd8afa-a3ca-4271-ba92-6d082e5c3b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:14.813023 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd8afa-a3ca-4271-ba92-6d082e5c3b01 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:14.813315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd8afa-a3ca-4271-ba92-6d082e5c3b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:14.813572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd8afa-a3ca-4271-ba92-6d082e5c3b01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:14.828276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b31ac-e934-42de-8f96-241a0be0366a None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:14.861559 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159198bb-646b-4d3f-b594-bedc7114ba19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:14.862157 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159198bb-646b-4d3f-b594-bedc7114ba19 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:14.862436 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159198bb-646b-4d3f-b594-bedc7114ba19 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:14.862709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159198bb-646b-4d3f-b594-bedc7114ba19 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:14.863405 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159198bb-646b-4d3f-b594-bedc7114ba19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:14.863709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159198bb-646b-4d3f-b594-bedc7114ba19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:14.865074 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159198bb-646b-4d3f-b594-bedc7114ba19 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:14.881119 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 266/520] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:14.881119 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159198bb-646b-4d3f-b594-bedc7114ba19 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:14.881341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd8afa-a3ca-4271-ba92-6d082e5c3b01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:14.881341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd8afa-a3ca-4271-ba92-6d082e5c3b01 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:14.944354 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:14.974398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:14.980902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159198bb-646b-4d3f-b594-bedc7114ba19 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:14.988035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 255/521] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:56:14.988860 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd8afa-a3ca-4271-ba92-6d082e5c3b01 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:15.017777 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f43aaf-5934-4b3e-8a5c-b3ce525b689e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:15.018372 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f43aaf-5934-4b3e-8a5c-b3ce525b689e None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:15.018652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f43aaf-5934-4b3e-8a5c-b3ce525b689e None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:15.018924 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f43aaf-5934-4b3e-8a5c-b3ce525b689e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:15.019712 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f43aaf-5934-4b3e-8a5c-b3ce525b689e None admin] RBAC: Adding query filter params (role.id=637ed3cd3d484839ab21de02c2bbe293, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=cce502a83f5343558c2136c0193e7d85) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:15.020507 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f43aaf-5934-4b3e-8a5c-b3ce525b689e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:15.021938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f43aaf-5934-4b3e-8a5c-b3ce525b689e None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:15.027160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 267/522] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:56:14 2022] GET /identity/v3/role_assignments?user.id=cce502a83f5343558c2136c0193e7d85&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=637ed3cd3d484839ab21de02c2bbe293 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:15.027916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f43aaf-5934-4b3e-8a5c-b3ce525b689e None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:15.098261 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd8afa-a3ca-4271-ba92-6d082e5c3b01 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:15.136418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e7921-a73b-4b68-b042-bd5a32cb86c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:15.137043 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e7921-a73b-4b68-b042-bd5a32cb86c6 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:15.137321 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e7921-a73b-4b68-b042-bd5a32cb86c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:15.137587 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12e7921-a73b-4b68-b042-bd5a32cb86c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:15.149361 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f43aaf-5934-4b3e-8a5c-b3ce525b689e None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:15.174582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:15.189179 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779baac5-7b1d-4bad-a5ed-18afb367b2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:15.189790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779baac5-7b1d-4bad-a5ed-18afb367b2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:15.190071 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779baac5-7b1d-4bad-a5ed-18afb367b2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:15.190346 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779baac5-7b1d-4bad-a5ed-18afb367b2a2 None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:15.191086 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779baac5-7b1d-4bad-a5ed-18afb367b2a2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:15.191405 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779baac5-7b1d-4bad-a5ed-18afb367b2a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:15.193365 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779baac5-7b1d-4bad-a5ed-18afb367b2a2 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:15.211757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 268/523] 10.222.0.36 () {56 vars in 1139 bytes} [Fri Sep 2 08:56:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:15.212892 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e7921-a73b-4b68-b042-bd5a32cb86c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:15.214346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12e7921-a73b-4b68-b042-bd5a32cb86c6 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:15.217107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779baac5-7b1d-4bad-a5ed-18afb367b2a2 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:15.237188 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:15.324206 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 256/524] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:56:15.324206 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e7921-a73b-4b68-b042-bd5a32cb86c6 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:15.329385 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779baac5-7b1d-4bad-a5ed-18afb367b2a2 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:15.371271 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bba692-7c68-4397-884f-437c7ea8afcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:15.372118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bba692-7c68-4397-884f-437c7ea8afcb None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:15.372395 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bba692-7c68-4397-884f-437c7ea8afcb None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:15.372663 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bba692-7c68-4397-884f-437c7ea8afcb None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:15.373295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bba692-7c68-4397-884f-437c7ea8afcb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:15.374743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bba692-7c68-4397-884f-437c7ea8afcb None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:15.393705 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:15.395933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 269/525] 10.222.0.36 () {60 vars in 1132 bytes} [Fri Sep 2 08:56:15 2022] POST /identity/v3/users => generated 247 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:56:15.397085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bba692-7c68-4397-884f-437c7ea8afcb None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:15.459498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12e7921-a73b-4b68-b042-bd5a32cb86c6 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:15.493859 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:15.507264 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff325bf-3dad-4fe6-a8f3-68de2f6ccad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:15.508564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff325bf-3dad-4fe6-a8f3-68de2f6ccad0 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:15.508903 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff325bf-3dad-4fe6-a8f3-68de2f6ccad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:15.509237 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff325bf-3dad-4fe6-a8f3-68de2f6ccad0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:15.525164 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff325bf-3dad-4fe6-a8f3-68de2f6ccad0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:15.526668 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff325bf-3dad-4fe6-a8f3-68de2f6ccad0 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:15.530045 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bba692-7c68-4397-884f-437c7ea8afcb None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:15.543480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-8ff325bf-3dad-4fe6-a8f3-68de2f6ccad0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:15.543480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:15.543480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:15.543480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:15.543480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:15.543480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:15.543480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:15.543480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:15.543480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:15.543480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:15.543480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:15.543480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:15.543480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:15.543480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:15.543480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:15.543480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:15.544435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:15.544435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:15.544435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:15.544435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:15.544435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:15.544435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:15.544435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:15.544435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:15.544435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:15.544435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:15.544435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:15.544435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:15.544435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:15.544435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:15.544435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:15.544435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:15.544435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:15.545557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:15.545557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:15.545557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:15.545557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 257/526] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:15.545769 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff325bf-3dad-4fe6-a8f3-68de2f6ccad0 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:15.579100 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929d09a-f60d-4e61-9e21-461f34b7dc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:15.579748 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929d09a-f60d-4e61-9e21-461f34b7dc61 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:15.580031 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929d09a-f60d-4e61-9e21-461f34b7dc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:15.580302 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929d09a-f60d-4e61-9e21-461f34b7dc61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:15.669324 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929d09a-f60d-4e61-9e21-461f34b7dc61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:15.670712 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929d09a-f60d-4e61-9e21-461f34b7dc61 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:15.676704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff325bf-3dad-4fe6-a8f3-68de2f6ccad0 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:15.695752 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:15.714012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6b72d-3a1c-4def-a196-add3935bb728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:15.714640 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6b72d-3a1c-4def-a196-add3935bb728 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:15.714915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6b72d-3a1c-4def-a196-add3935bb728 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:15.715297 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6b72d-3a1c-4def-a196-add3935bb728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:15.727594 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6b72d-3a1c-4def-a196-add3935bb728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:15.729063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6b72d-3a1c-4def-a196-add3935bb728 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:15.744148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-14f6b72d-3a1c-4def-a196-add3935bb728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:15.744148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:15.744148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:15.744148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:15.744148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:15.744148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:15.744148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:15.744148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:15.744148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:15.744148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:15.744148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:15.744148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:15.744148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:15.744148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:15.744148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:15.744148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:15.745103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:15.745103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:15.745103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:15.745103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:15.745103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:15.745103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:15.745103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:15.745103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:15.745103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:15.745103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:15.745103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:15.745103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:15.745103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:15.745103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:15.745103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:15.745103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:15.745103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:15.745955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:15.745955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:15.745955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:15.745955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 258/527] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:15.746171 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6b72d-3a1c-4def-a196-add3935bb728 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:15.748167 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:15.773696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 270/528] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:56:15.774635 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929d09a-f60d-4e61-9e21-461f34b7dc61 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:15.848759 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6b72d-3a1c-4def-a196-add3935bb728 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:15.875540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929d09a-f60d-4e61-9e21-461f34b7dc61 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:15.883600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcce15-8137-40a2-b17f-2af9f03b74ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:15.884206 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcce15-8137-40a2-b17f-2af9f03b74ab None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:15.884481 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcce15-8137-40a2-b17f-2af9f03b74ab None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:15.884752 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdcce15-8137-40a2-b17f-2af9f03b74ab None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:15.885498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdcce15-8137-40a2-b17f-2af9f03b74ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:15.885801 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdcce15-8137-40a2-b17f-2af9f03b74ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:15.887272 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdcce15-8137-40a2-b17f-2af9f03b74ab None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:15.899491 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 259/529] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:15.900309 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcce15-8137-40a2-b17f-2af9f03b74ab None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:15.910038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e58ee2-38b0-4caf-8733-4284aaaf41db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:15.911215 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e58ee2-38b0-4caf-8733-4284aaaf41db None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:15.911495 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e58ee2-38b0-4caf-8733-4284aaaf41db None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:15.911763 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e58ee2-38b0-4caf-8733-4284aaaf41db None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:15.912407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e58ee2-38b0-4caf-8733-4284aaaf41db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:15.912706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e58ee2-38b0-4caf-8733-4284aaaf41db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:15.914071 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e58ee2-38b0-4caf-8733-4284aaaf41db None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:15.926544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 271/530] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:15.927389 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e58ee2-38b0-4caf-8733-4284aaaf41db None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:15.937369 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:15.963429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:16.012501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdcce15-8137-40a2-b17f-2af9f03b74ab None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:16.029265 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e58ee2-38b0-4caf-8733-4284aaaf41db None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:16.048507 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b42d18-b8db-4973-bbaa-f52e7eb4060d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:16.049360 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b42d18-b8db-4973-bbaa-f52e7eb4060d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:16.049779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b42d18-b8db-4973-bbaa-f52e7eb4060d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:16.050222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b42d18-b8db-4973-bbaa-f52e7eb4060d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:16.061971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b42d18-b8db-4973-bbaa-f52e7eb4060d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:16.063887 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b42d18-b8db-4973-bbaa-f52e7eb4060d None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:16.064721 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645466e-e676-441a-aaec-2e90cf43f322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:16.065421 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645466e-e676-441a-aaec-2e90cf43f322 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:16.065821 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645466e-e676-441a-aaec-2e90cf43f322 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:16.066162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645466e-e676-441a-aaec-2e90cf43f322 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:16.073088 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645466e-e676-441a-aaec-2e90cf43f322 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:16.074610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4645466e-e676-441a-aaec-2e90cf43f322 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:16.080942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-23b42d18-b8db-4973-bbaa-f52e7eb4060d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:16.080942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:16.080942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:16.080942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:16.080942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:16.080942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:16.080942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:16.080942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:16.080942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:16.080942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:16.080942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:16.080942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:16.080942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:16.080942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:16.080942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:16.080942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:16.081854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:16.081854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:16.081854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:16.081854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:16.081854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:16.081854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:16.081854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:16.081854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:16.081854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:16.081854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:16.081854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:16.081854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:16.081854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:16.081854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:16.081854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:16.081854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:16.081854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:16.083458 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:16.083458 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:16.083458 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:16.085109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 260/531] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:16.086072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b42d18-b8db-4973-bbaa-f52e7eb4060d None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:16.088019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-4645466e-e676-441a-aaec-2e90cf43f322 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:16.088019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:16.088019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:16.088019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:16.088019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:16.088019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:16.088019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:16.088019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:16.088019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:16.088019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:16.088019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:16.088019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:16.088019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:16.088019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:16.088019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:16.088855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:16.088855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:16.088855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:16.088855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:16.088855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:16.088855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:16.088855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:16.088855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:16.088855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:16.088855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:16.088855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:16.088855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:16.088855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:16.088855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:16.088855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:16.088855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:16.088855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:16.089703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:16.089703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:16.089703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:16.089703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:16.089703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:16.089703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:16.089703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:16.089703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:16.094351 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 272/532] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:16.095372 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645466e-e676-441a-aaec-2e90cf43f322 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:16.180763 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:16.202297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645466e-e676-441a-aaec-2e90cf43f322 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:16.207200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b42d18-b8db-4973-bbaa-f52e7eb4060d None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:16.211554 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:16.236691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a984ac3-d661-472e-9ae4-2a1783cf74e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:16.237439 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a984ac3-d661-472e-9ae4-2a1783cf74e9 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:16.237841 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a984ac3-d661-472e-9ae4-2a1783cf74e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:16.238212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a984ac3-d661-472e-9ae4-2a1783cf74e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:16.239490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a984ac3-d661-472e-9ae4-2a1783cf74e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:16.239969 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a984ac3-d661-472e-9ae4-2a1783cf74e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:16.241545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a984ac3-d661-472e-9ae4-2a1783cf74e9 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:16.249264 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bb1cd-b81a-433c-a6b3-f1d484f27135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:16.250076 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bb1cd-b81a-433c-a6b3-f1d484f27135 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:16.250495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bb1cd-b81a-433c-a6b3-f1d484f27135 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:16.251383 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bb1cd-b81a-433c-a6b3-f1d484f27135 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:16.259148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86bb1cd-b81a-433c-a6b3-f1d484f27135 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:16.259567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 273/533] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:16.260540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a984ac3-d661-472e-9ae4-2a1783cf74e9 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:16.261067 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86bb1cd-b81a-433c-a6b3-f1d484f27135 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:16.275344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-a86bb1cd-b81a-433c-a6b3-f1d484f27135 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:16.275344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:16.275344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:16.275344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:16.275344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:16.275344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:16.275344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:16.275344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:16.275344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:16.275344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:16.275344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:16.275344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:16.275344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:16.275344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:16.275344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:16.276222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:16.276222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:16.276222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:16.276222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:16.276222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:16.276222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:16.276222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:16.276222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:16.276222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:16.276222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:16.276222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:16.276222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:16.276222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:16.276222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:16.276222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:16.276222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:16.276222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:16.277052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:16.277052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:16.277052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:16.277052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:16.277052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:16.277052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:16.277052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:16.277052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:16.279634 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 261/534] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:16.280866 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bb1cd-b81a-433c-a6b3-f1d484f27135 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:16.396025 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a984ac3-d661-472e-9ae4-2a1783cf74e9 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:16.399207 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bb1cd-b81a-433c-a6b3-f1d484f27135 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:16.431768 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85df03-bace-4230-99b3-6140638c684d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:16.432693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85df03-bace-4230-99b3-6140638c684d None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:16.433210 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85df03-bace-4230-99b3-6140638c684d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:16.433630 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85df03-bace-4230-99b3-6140638c684d None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:16.434329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce13774-d26e-4a9c-8f2a-a2a0e7471e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:16.434876 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85df03-bace-4230-99b3-6140638c684d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:16.435370 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85df03-bace-4230-99b3-6140638c684d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:16.435569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce13774-d26e-4a9c-8f2a-a2a0e7471e0a None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:16.435941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce13774-d26e-4a9c-8f2a-a2a0e7471e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:16.436304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce13774-d26e-4a9c-8f2a-a2a0e7471e0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:16.437373 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85df03-bace-4230-99b3-6140638c684d None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:16.444032 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce13774-d26e-4a9c-8f2a-a2a0e7471e0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:16.445602 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce13774-d26e-4a9c-8f2a-a2a0e7471e0a None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:16.450926 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 274/535] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:16.451937 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85df03-bace-4230-99b3-6140638c684d None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:16.462435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-5ce13774-d26e-4a9c-8f2a-a2a0e7471e0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:16.462435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:16.462435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:16.462435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:16.462435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:16.462435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:16.462435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:16.462435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:16.462435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:16.462435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:16.462435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:16.462435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:16.462435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:16.462435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:16.462435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:16.463479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:16.463479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:16.463479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:16.463479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:16.463479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:16.463479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:16.463479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:16.463479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:16.463479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:16.463479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:16.463479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:16.463479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:16.463479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:16.463479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:16.463479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:16.463479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:16.463479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:16.464496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:16.464496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:16.464496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:16.464496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:16.464496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:16.464496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:16.464496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:16.464496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:16.467160 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 262/536] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:16.468020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce13774-d26e-4a9c-8f2a-a2a0e7471e0a None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:16.471723 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:16.477326 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:16.579542 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85df03-bace-4230-99b3-6140638c684d None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:16.585245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce13774-d26e-4a9c-8f2a-a2a0e7471e0a None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:16.618825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfe1c5-682b-420a-960d-4a9eaf64d7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:16.619500 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfe1c5-682b-420a-960d-4a9eaf64d7a7 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:16.619776 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfe1c5-682b-420a-960d-4a9eaf64d7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:16.620045 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddfe1c5-682b-420a-960d-4a9eaf64d7a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:16.620861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfe1c5-682b-420a-960d-4a9eaf64d7a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:16.621178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfe1c5-682b-420a-960d-4a9eaf64d7a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:16.622581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bf934-cca1-4fc3-af05-18f726838385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:16.622805 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddfe1c5-682b-420a-960d-4a9eaf64d7a7 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:16.623653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bf934-cca1-4fc3-af05-18f726838385 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:16.624138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bf934-cca1-4fc3-af05-18f726838385 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:16.624601 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bf934-cca1-4fc3-af05-18f726838385 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:16.625564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bf934-cca1-4fc3-af05-18f726838385 None admin] RBAC: Adding query filter params (role.id=637ed3cd3d484839ab21de02c2bbe293, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=f8370135a78f414f9184c883ccb92767) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:16.626073 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bf934-cca1-4fc3-af05-18f726838385 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:16.627759 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bf934-cca1-4fc3-af05-18f726838385 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:16.633974 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 263/537] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:56:16 2022] GET /identity/v3/role_assignments?user.id=f8370135a78f414f9184c883ccb92767&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=637ed3cd3d484839ab21de02c2bbe293 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:16.634986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bf934-cca1-4fc3-af05-18f726838385 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:16.637878 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 275/538] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:16.643058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfe1c5-682b-420a-960d-4a9eaf64d7a7 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:16.709029 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:16.752600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bf934-cca1-4fc3-af05-18f726838385 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:16.762087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:16.797176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddfe1c5-682b-420a-960d-4a9eaf64d7a7 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:16.815846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a565c61-0155-409f-8618-2f5781e0a382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:16.816761 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a565c61-0155-409f-8618-2f5781e0a382 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:16.817182 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a565c61-0155-409f-8618-2f5781e0a382 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:16.817581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a565c61-0155-409f-8618-2f5781e0a382 None admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:16.818449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a565c61-0155-409f-8618-2f5781e0a382 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:16.818959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a565c61-0155-409f-8618-2f5781e0a382 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:16.820547 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a565c61-0155-409f-8618-2f5781e0a382 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:16.852640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2a1e6-d257-4e93-9f7d-895b8f2fbb18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:16.853511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2a1e6-d257-4e93-9f7d-895b8f2fbb18 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:16.853959 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2a1e6-d257-4e93-9f7d-895b8f2fbb18 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:16.854429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2a1e6-d257-4e93-9f7d-895b8f2fbb18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:16.855385 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2a1e6-d257-4e93-9f7d-895b8f2fbb18 None admin] RBAC: Adding query filter params (role.id=637ed3cd3d484839ab21de02c2bbe293, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=2474d89f10764afbbc5e740dccaae4df) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:16.855842 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2a1e6-d257-4e93-9f7d-895b8f2fbb18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:16.857688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2a1e6-d257-4e93-9f7d-895b8f2fbb18 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:16.862122 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 264/539] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:16.869390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a565c61-0155-409f-8618-2f5781e0a382 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:16.876306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 276/540] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:56:16 2022] GET /identity/v3/role_assignments?user.id=2474d89f10764afbbc5e740dccaae4df&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=637ed3cd3d484839ab21de02c2bbe293 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:17.000256 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a565c61-0155-409f-8618-2f5781e0a382 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:17.000942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae392362-1c99-434b-b82b-e2ac7e6ef5d7 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:17.001443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae392362-1c99-434b-b82b-e2ac7e6ef5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:17.002106 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae392362-1c99-434b-b82b-e2ac7e6ef5d7 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:17.004969 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 277/541] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:56:17.018365 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03af1e7-757a-4758-8496-a450370d124e None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:17.018645 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03af1e7-757a-4758-8496-a450370d124e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:17.018929 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03af1e7-757a-4758-8496-a450370d124e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:17.036048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:17.045171 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579994a-2600-499c-8b6b-88403319fca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:17.045780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579994a-2600-499c-8b6b-88403319fca0 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:17.046059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579994a-2600-499c-8b6b-88403319fca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:17.046331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579994a-2600-499c-8b6b-88403319fca0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:17.047053 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579994a-2600-499c-8b6b-88403319fca0 None admin] RBAC: Adding query filter params (role.id=637ed3cd3d484839ab21de02c2bbe293, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=07a825fddbc04f918bcd631976038cf9) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:17.047384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579994a-2600-499c-8b6b-88403319fca0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:17.048512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-c03af1e7-757a-4758-8496-a450370d124e None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:17.048801 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579994a-2600-499c-8b6b-88403319fca0 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:17.049780 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c03af1e7-757a-4758-8496-a450370d124e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:17.054847 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 265/542] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:56:16 2022] GET /identity/v3/role_assignments?user.id=07a825fddbc04f918bcd631976038cf9&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=637ed3cd3d484839ab21de02c2bbe293 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:17.140829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 278/543] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:17.493121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579994a-2600-499c-8b6b-88403319fca0 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:17.640249 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579994a-2600-499c-8b6b-88403319fca0 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:17.678503 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a90bf3-1719-4b70-8cde-cb1144b10b32 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:17.679010 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a90bf3-1719-4b70-8cde-cb1144b10b32 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:17.679259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a90bf3-1719-4b70-8cde-cb1144b10b32 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:17.681066 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 279/544] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:17.683634 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9717a95-0daf-4127-8ab1-939bd215ac89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:17.684735 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9717a95-0daf-4127-8ab1-939bd215ac89 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:17.685186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9717a95-0daf-4127-8ab1-939bd215ac89 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:17.685632 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9717a95-0daf-4127-8ab1-939bd215ac89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:17.686617 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf873e10-2198-45cd-a2f8-e0361f6dd785 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:17.686980 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf873e10-2198-45cd-a2f8-e0361f6dd785 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:17.687362 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf873e10-2198-45cd-a2f8-e0361f6dd785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:17.705109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9717a95-0daf-4127-8ab1-939bd215ac89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:17.715238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-e9717a95-0daf-4127-8ab1-939bd215ac89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:17.715238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:17.715238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:17.715238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:17.715238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:17.715238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:17.715238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:17.715238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:17.715238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:17.715238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:17.715238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:17.715238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:17.715238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:17.715238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:17.715238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:17.715238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:17.716125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:17.716125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:17.716125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:17.716125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:17.716125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:17.716125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:17.716125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:17.716125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:17.716125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:17.716125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:17.716125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:17.716125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:17.716125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:17.716125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:17.716125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:17.716125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:17.716907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:17.716907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:17.716907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:17.716907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:17.716907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:17.716907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:17.716907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:17.716907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:17.716907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:17.716907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:17.716907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:17.716907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:17.723976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 266/545] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:56:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:17.725066 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-cf873e10-2198-45cd-a2f8-e0361f6dd785 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:17.726460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9717a95-0daf-4127-8ab1-939bd215ac89 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:17.727765 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-cf873e10-2198-45cd-a2f8-e0361f6dd785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:17.787489 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:17.841856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9717a95-0daf-4127-8ab1-939bd215ac89 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:17.849576 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:17.861769 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 280/546] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:56:17.878815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253aeb8-a249-4d3f-b63c-68171eca19cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:17.879715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253aeb8-a249-4d3f-b63c-68171eca19cc None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:17.880137 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253aeb8-a249-4d3f-b63c-68171eca19cc None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:17.880552 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253aeb8-a249-4d3f-b63c-68171eca19cc None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:17.881422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253aeb8-a249-4d3f-b63c-68171eca19cc None admin] RBAC: Adding query filter params (name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:17.881864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253aeb8-a249-4d3f-b63c-68171eca19cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:17.883444 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9253aeb8-a249-4d3f-b63c-68171eca19cc None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:17.892529 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 267/547] 10.222.0.36 () {56 vars in 1103 bytes} [Fri Sep 2 08:56:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:17.898733 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf873e10-2198-45cd-a2f8-e0361f6dd785 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:18.043766 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf873e10-2198-45cd-a2f8-e0361f6dd785 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:18.090112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b5afa-bd81-48e5-b6f2-6c4c39298d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:18.090746 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b5afa-bd81-48e5-b6f2-6c4c39298d19 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:18.091025 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b5afa-bd81-48e5-b6f2-6c4c39298d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:18.091325 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b5afa-bd81-48e5-b6f2-6c4c39298d19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:18.107846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b5afa-bd81-48e5-b6f2-6c4c39298d19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:18.109338 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b5afa-bd81-48e5-b6f2-6c4c39298d19 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:18.128585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-4b2b5afa-bd81-48e5-b6f2-6c4c39298d19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:18.128585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:18.128585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:18.128585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:18.128585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:18.128585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:18.128585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:18.128585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:18.128585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:18.128585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:18.128585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:18.128585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:18.128585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:18.128585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:18.128585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:18.128585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:18.130027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:18.130027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:18.130027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:18.130027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:18.130027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:18.130027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:18.130027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:18.130027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:18.130027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:18.130027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:18.130027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:18.130027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:18.130027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:18.130027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:18.130027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:18.130027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:18.130027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:18.130851 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:18.130851 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:18.130851 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:18.130851 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 281/548] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:18.135839 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253aeb8-a249-4d3f-b63c-68171eca19cc None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:18.236640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b5afa-bd81-48e5-b6f2-6c4c39298d19 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:18.261327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253aeb8-a249-4d3f-b63c-68171eca19cc None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:18.295021 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:18.304175 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2abe5-542f-45a4-8ea5-a6c9a66aa10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:18.305085 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2abe5-542f-45a4-8ea5-a6c9a66aa10f None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:18.305714 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2abe5-542f-45a4-8ea5-a6c9a66aa10f None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:18.306648 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2abe5-542f-45a4-8ea5-a6c9a66aa10f None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:18.308238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2abe5-542f-45a4-8ea5-a6c9a66aa10f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:18.308956 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2abe5-542f-45a4-8ea5-a6c9a66aa10f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:18.311379 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe2abe5-542f-45a4-8ea5-a6c9a66aa10f None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:18.337114 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 268/549] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:18.342290 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2abe5-542f-45a4-8ea5-a6c9a66aa10f None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:18.353175 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:18.449142 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b5afa-bd81-48e5-b6f2-6c4c39298d19 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:18.494345 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb03295-dcf1-4b28-9ea9-0d681986a11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:18.494891 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb03295-dcf1-4b28-9ea9-0d681986a11b None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:18.495221 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb03295-dcf1-4b28-9ea9-0d681986a11b None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:18.495507 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb03295-dcf1-4b28-9ea9-0d681986a11b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:18.497611 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe2abe5-542f-45a4-8ea5-a6c9a66aa10f None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:18.506684 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb03295-dcf1-4b28-9ea9-0d681986a11b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:18.512137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-ecb03295-dcf1-4b28-9ea9-0d681986a11b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:18.512137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:18.512137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:18.512137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:18.512137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:18.512137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:18.512137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:18.512137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:18.512137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:18.512137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:18.512137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:18.512137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:18.512137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:18.512137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:18.512137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:18.512137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:18.513685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:18.513685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:18.513685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:18.513685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:18.513685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:18.513685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:18.513685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:18.513685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:18.513685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:18.513685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:18.513685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:18.513685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:18.513685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:18.513685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:18.513685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:18.513685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:18.514671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:18.514671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:18.514671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:18.514671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:18.514671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:18.514671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:18.514671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:18.514671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:18.514671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:18.514671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:18.514671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:18.514671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:18.514671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 282/550] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:56:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:18.527133 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb03295-dcf1-4b28-9ea9-0d681986a11b None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:18.572251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7a61f-f03b-4a96-a54e-6b8db9e339f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:18.573248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7a61f-f03b-4a96-a54e-6b8db9e339f8 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:18.573759 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7a61f-f03b-4a96-a54e-6b8db9e339f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:18.574242 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7a61f-f03b-4a96-a54e-6b8db9e339f8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:18.585604 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7a61f-f03b-4a96-a54e-6b8db9e339f8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:18.587359 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7a61f-f03b-4a96-a54e-6b8db9e339f8 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:18.601468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-4ef7a61f-f03b-4a96-a54e-6b8db9e339f8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 08:56:18.601468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:18.601468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:18.601468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:18.601468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:18.601468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:18.601468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:18.601468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:18.601468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:18.601468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:18.601468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:18.601468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:18.601468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:18.601468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:18.601468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:18.601468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:18.602422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:18.602422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:18.602422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:18.602422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:18.602422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:18.602422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:18.602422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:18.602422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:18.602422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:18.602422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:18.602422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:18.602422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:18.602422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:18.602422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:18.602422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:18.602422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:18.602422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:18.602422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:18.603557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:18.603557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:18.603557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:18.603557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:18.603557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:18.603557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:18.603557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:18.603557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:18.603557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 08:56:18.603557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:18.606004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 269/551] 10.222.0.36 () {56 vars in 1093 bytes} [Fri Sep 2 08:56:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:18.611638 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7a61f-f03b-4a96-a54e-6b8db9e339f8 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:18.632362 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb03295-dcf1-4b28-9ea9-0d681986a11b None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:18.659656 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:18.659855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:18.672478 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc7731-08c2-435e-b78f-923484c407fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:18.673413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc7731-08c2-435e-b78f-923484c407fe None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:18.677400 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc7731-08c2-435e-b78f-923484c407fe None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:18.677895 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc7731-08c2-435e-b78f-923484c407fe None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:18.678829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc7731-08c2-435e-b78f-923484c407fe None admin] RBAC: Adding query filter params (name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:18.679902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc7731-08c2-435e-b78f-923484c407fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:18.681613 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc7731-08c2-435e-b78f-923484c407fe None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:18.698462 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 283/552] 10.222.0.36 () {56 vars in 1103 bytes} [Fri Sep 2 08:56:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:18.706262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc7731-08c2-435e-b78f-923484c407fe None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:18.776591 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7a61f-f03b-4a96-a54e-6b8db9e339f8 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:18.849556 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb5353-82fb-4ca5-bfd3-2ed9371c232a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:18.850549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb5353-82fb-4ca5-bfd3-2ed9371c232a None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:18.855602 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb5353-82fb-4ca5-bfd3-2ed9371c232a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:18.855808 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb5353-82fb-4ca5-bfd3-2ed9371c232a None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:18.856535 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb5353-82fb-4ca5-bfd3-2ed9371c232a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:18.856855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb5353-82fb-4ca5-bfd3-2ed9371c232a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:18.858321 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb5353-82fb-4ca5-bfd3-2ed9371c232a None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:18.876966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 270/553] 10.222.0.36 () {56 vars in 1133 bytes} [Fri Sep 2 08:56:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:18.888187 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb5353-82fb-4ca5-bfd3-2ed9371c232a None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:18.889403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc7731-08c2-435e-b78f-923484c407fe None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:18.926551 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb48f9-2e1e-4526-aee8-08e9f8ffd093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:18.927636 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb48f9-2e1e-4526-aee8-08e9f8ffd093 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:18.928122 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb48f9-2e1e-4526-aee8-08e9f8ffd093 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:18.928595 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb48f9-2e1e-4526-aee8-08e9f8ffd093 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:18.940428 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb48f9-2e1e-4526-aee8-08e9f8ffd093 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:18.942413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb48f9-2e1e-4526-aee8-08e9f8ffd093 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:18.957911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-c6cb48f9-2e1e-4526-aee8-08e9f8ffd093 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:18.957911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:18.957911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:18.957911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:18.957911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:18.957911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:18.957911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:18.957911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:18.957911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:18.957911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:18.957911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:18.957911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:18.957911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:18.957911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:18.957911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:18.957911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:18.958880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:18.958880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:18.958880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:18.958880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:18.958880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:18.958880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:18.958880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:18.958880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:18.958880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:18.958880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:18.958880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:18.958880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:18.958880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:18.958880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:18.958880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:18.958880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:18.958880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:18.959816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:18.959816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:18.959816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:18.961861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 284/554] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:18.967708 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb48f9-2e1e-4526-aee8-08e9f8ffd093 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:18.997369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:19.080359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb48f9-2e1e-4526-aee8-08e9f8ffd093 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:19.101797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:19.119781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c94b45-6ff2-4240-8745-edbca59962ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:19.120854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c94b45-6ff2-4240-8745-edbca59962ed None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:19.121411 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c94b45-6ff2-4240-8745-edbca59962ed None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:19.121884 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c94b45-6ff2-4240-8745-edbca59962ed None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:19.122746 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c94b45-6ff2-4240-8745-edbca59962ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:19.123276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c94b45-6ff2-4240-8745-edbca59962ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:19.124869 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c94b45-6ff2-4240-8745-edbca59962ed None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:19.129247 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb5353-82fb-4ca5-bfd3-2ed9371c232a None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:19.138537 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 285/555] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:19.144142 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c94b45-6ff2-4240-8745-edbca59962ed None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:19.220136 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68577e-892d-44c9-be84-fbf98e59595b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:19.220775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68577e-892d-44c9-be84-fbf98e59595b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:19.221050 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68577e-892d-44c9-be84-fbf98e59595b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:19.221318 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68577e-892d-44c9-be84-fbf98e59595b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:19.232495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68577e-892d-44c9-be84-fbf98e59595b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:19.234626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68577e-892d-44c9-be84-fbf98e59595b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:19.235468 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:19.253407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-4b68577e-892d-44c9-be84-fbf98e59595b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:19.253407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:19.253407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:19.253407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:19.253407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:19.253407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:19.253407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:19.253407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:19.253407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:19.253407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:19.253407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:19.253407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:19.253407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:19.253407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:19.253407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:19.253407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:19.254832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:19.254832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:19.254832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:19.254832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:19.254832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:19.254832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:19.254832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:19.254832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:19.254832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:19.254832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:19.254832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:19.254832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:19.254832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:19.254832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:19.254832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:19.254832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:19.254832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:19.255746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:19.255746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:19.255746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:19.255931 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c94b45-6ff2-4240-8745-edbca59962ed None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:19.256018 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 271/556] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:19.256525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1504a-c9d3-41ca-9781-4347a411ab57 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:19.256623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1504a-c9d3-41ca-9781-4347a411ab57 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:19.256888 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1504a-c9d3-41ca-9781-4347a411ab57 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:19.258263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 272/557] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:19.259171 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cccb4d-8394-4cb7-acf3-e67eb846b8e0 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:19.259450 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cccb4d-8394-4cb7-acf3-e67eb846b8e0 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:19.259776 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cccb4d-8394-4cb7-acf3-e67eb846b8e0 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:19.265198 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 273/558] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:19.265198 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cccb4d-8394-4cb7-acf3-e67eb846b8e0 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:19.300996 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f657c-836b-452b-b9f9-096c7920c3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:19.301758 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f657c-836b-452b-b9f9-096c7920c3c2 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:19.301929 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f657c-836b-452b-b9f9-096c7920c3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:19.302199 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f657c-836b-452b-b9f9-096c7920c3c2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:19.309807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f657c-836b-452b-b9f9-096c7920c3c2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:19.311291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f657c-836b-452b-b9f9-096c7920c3c2 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:19.322744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-0b5f657c-836b-452b-b9f9-096c7920c3c2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 02 08:56:19.322744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:19.322744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:19.322744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:19.322744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:19.322744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:19.322744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:19.322744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:19.322744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:19.322744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:19.322744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:19.322744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:19.322744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:19.322744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:19.322744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:19.322744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:19.323718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:19.323718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:19.323718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:19.323718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:19.323718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:19.323718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:19.323718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:19.323718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:19.323718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:19.323718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:19.323718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:19.323718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:19.323718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:19.323718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:19.323718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:19.323718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:19.323718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:19.323718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:19.324740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:19.324740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:19.324740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:19.324740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:19.324740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:19.324740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:19.324740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:19.324740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:19.324740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 02 08:56:19.324740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:19.324740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 286/559] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:56:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:19.325341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed835cca-f3f0-44fd-a7c9-795c1ed43197 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:19.325932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed835cca-f3f0-44fd-a7c9-795c1ed43197 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:19.325932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed835cca-f3f0-44fd-a7c9-795c1ed43197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:19.346780 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-ed835cca-f3f0-44fd-a7c9-795c1ed43197 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:19.348477 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-ed835cca-f3f0-44fd-a7c9-795c1ed43197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:19.381845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cccb4d-8394-4cb7-acf3-e67eb846b8e0 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:19.423109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 287/560] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:19.423293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d11ea-e72d-46ee-bf35-54e9635f4f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:19.423293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d11ea-e72d-46ee-bf35-54e9635f4f93 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:19.423293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d11ea-e72d-46ee-bf35-54e9635f4f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:19.423537 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37b540-d409-478d-8420-147fe6d67335 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:19.423609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d11ea-e72d-46ee-bf35-54e9635f4f93 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:19.423677 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37b540-d409-478d-8420-147fe6d67335 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:19.423677 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d37b540-d409-478d-8420-147fe6d67335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:19.423797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d11ea-e72d-46ee-bf35-54e9635f4f93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:19.423797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d11ea-e72d-46ee-bf35-54e9635f4f93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:19.423797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d11ea-e72d-46ee-bf35-54e9635f4f93 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:19.442204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 274/561] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:19.443152 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d11ea-e72d-46ee-bf35-54e9635f4f93 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:19.445822 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:19.453162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-4d37b540-d409-478d-8420-147fe6d67335 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:19.454430 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-4d37b540-d409-478d-8420-147fe6d67335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:19.529140 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 288/562] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:19.530545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeed854-955f-4105-9d3c-d6260b3b85dc None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:19.530832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeed854-955f-4105-9d3c-d6260b3b85dc None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:19.535478 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeed854-955f-4105-9d3c-d6260b3b85dc None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:19.535478 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 289/563] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:19.535478 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeed854-955f-4105-9d3c-d6260b3b85dc None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:19.540988 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:19.556771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d11ea-e72d-46ee-bf35-54e9635f4f93 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:19.601019 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a77ac6-5bcb-4b67-8846-c5121f537e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:19.602038 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a77ac6-5bcb-4b67-8846-c5121f537e1c None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:19.602757 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a77ac6-5bcb-4b67-8846-c5121f537e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:19.603245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a77ac6-5bcb-4b67-8846-c5121f537e1c None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:19.604235 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a77ac6-5bcb-4b67-8846-c5121f537e1c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:19.604722 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a77ac6-5bcb-4b67-8846-c5121f537e1c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:19.607037 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a77ac6-5bcb-4b67-8846-c5121f537e1c None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:19.629938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 275/564] 10.222.0.36 () {56 vars in 1135 bytes} [Fri Sep 2 08:56:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:19.631855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b164408-ee5d-4b80-9685-3a83b510f500 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:19.632347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b164408-ee5d-4b80-9685-3a83b510f500 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:19.633117 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b164408-ee5d-4b80-9685-3a83b510f500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:19.653703 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-1b164408-ee5d-4b80-9685-3a83b510f500 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:19.655120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1b164408-ee5d-4b80-9685-3a83b510f500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:19.663150 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeed854-955f-4105-9d3c-d6260b3b85dc None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:19.709970 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b13ffc-392e-43b5-8de7-ebff41b70a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:19.710602 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b13ffc-392e-43b5-8de7-ebff41b70a86 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:19.710922 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b13ffc-392e-43b5-8de7-ebff41b70a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:19.711235 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b13ffc-392e-43b5-8de7-ebff41b70a86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:19.716644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:19.718378 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b13ffc-392e-43b5-8de7-ebff41b70a86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:19.720104 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b13ffc-392e-43b5-8de7-ebff41b70a86 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:19.729692 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 276/565] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:19.730496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b164408-ee5d-4b80-9685-3a83b510f500 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:19.733225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-c5b13ffc-392e-43b5-8de7-ebff41b70a86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:19.733225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:19.733225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:19.733225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:19.733225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:19.733225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:19.733225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:19.733225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:19.733225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:19.733225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:19.733225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:19.733225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:19.733225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:19.733225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:19.733225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:19.734108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:19.734108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:19.734108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:19.734108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:19.734108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:19.734108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:19.734108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:19.734108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:19.734108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:19.734108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:19.734108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:19.734108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:19.734108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:19.734108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:19.734108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:19.734108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:19.734108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:19.739659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:19.739659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:19.739659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:19.739659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:19.739659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:19.739659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:19.739659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:19.739659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:19.739659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 290/566] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:19.739659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b13ffc-392e-43b5-8de7-ebff41b70a86 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:19.806551 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:19.846986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b164408-ee5d-4b80-9685-3a83b510f500 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:19.850243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b13ffc-392e-43b5-8de7-ebff41b70a86 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:19.882038 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099789b-83e7-4ec6-92c9-d5c83e6f01f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:19.882647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099789b-83e7-4ec6-92c9-d5c83e6f01f9 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:19.882930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099789b-83e7-4ec6-92c9-d5c83e6f01f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:19.883235 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e099789b-83e7-4ec6-92c9-d5c83e6f01f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:19.885864 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95082471-d880-4809-97d0-9e8d289556fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:19.886594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95082471-d880-4809-97d0-9e8d289556fd None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:19.886867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95082471-d880-4809-97d0-9e8d289556fd None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:19.887234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95082471-d880-4809-97d0-9e8d289556fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:19.897665 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95082471-d880-4809-97d0-9e8d289556fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:19.902729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-95082471-d880-4809-97d0-9e8d289556fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:19.902729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:19.902729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:19.902729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:19.902729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:19.902729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:19.902729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:19.902729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:19.902729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:19.902729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:19.902729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:19.902729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:19.902729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:19.902729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:19.902729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:19.902729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:19.903693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:19.903693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:19.903693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:19.903693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:19.903693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:19.903693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:19.903693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:19.903693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:19.903693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:19.903693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:19.903693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:19.903693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:19.903693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:19.903693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:19.903693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:19.903693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:19.904532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:19.904532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:19.904532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:19.904532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:19.904532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:19.904532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:19.904532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:19.904532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:19.904532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:19.904532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:19.904532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:19.904532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:19.904532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 291/567] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:56:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:19.905228 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95082471-d880-4809-97d0-9e8d289556fd None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:19.946003 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:19.957537 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099789b-83e7-4ec6-92c9-d5c83e6f01f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:19.958997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e099789b-83e7-4ec6-92c9-d5c83e6f01f9 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:20.008319 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95082471-d880-4809-97d0-9e8d289556fd None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:20.031626 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:20.042619 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964b7f2-0f44-45e3-86e2-03414405a8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:20.043250 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964b7f2-0f44-45e3-86e2-03414405a8ae None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:20.043524 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964b7f2-0f44-45e3-86e2-03414405a8ae None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:20.043824 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964b7f2-0f44-45e3-86e2-03414405a8ae None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:20.044504 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964b7f2-0f44-45e3-86e2-03414405a8ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:20.044811 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964b7f2-0f44-45e3-86e2-03414405a8ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:20.046233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964b7f2-0f44-45e3-86e2-03414405a8ae None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:20.058477 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 292/568] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:20.059207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964b7f2-0f44-45e3-86e2-03414405a8ae None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:20.064265 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 277/569] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:56:20.064265 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099789b-83e7-4ec6-92c9-d5c83e6f01f9 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:20.149220 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:20.159803 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e099789b-83e7-4ec6-92c9-d5c83e6f01f9 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:20.161165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964b7f2-0f44-45e3-86e2-03414405a8ae None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:20.194359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57782f0-e331-463a-9660-111fc2c1d7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:20.195007 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57782f0-e331-463a-9660-111fc2c1d7cd None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:20.195340 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74d174-642a-4855-b5c7-9ffb2b17c2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:20.195474 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57782f0-e331-463a-9660-111fc2c1d7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:20.195640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57782f0-e331-463a-9660-111fc2c1d7cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:20.195937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74d174-642a-4855-b5c7-9ffb2b17c2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:20.196210 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74d174-642a-4855-b5c7-9ffb2b17c2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:20.196475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74d174-642a-4855-b5c7-9ffb2b17c2b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:20.197124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74d174-642a-4855-b5c7-9ffb2b17c2b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:20.197422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74d174-642a-4855-b5c7-9ffb2b17c2b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:20.198783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74d174-642a-4855-b5c7-9ffb2b17c2b1 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:20.205617 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57782f0-e331-463a-9660-111fc2c1d7cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:20.206899 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 278/570] 10.222.0.36 () {56 vars in 1103 bytes} [Fri Sep 2 08:56:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:20.207574 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74d174-642a-4855-b5c7-9ffb2b17c2b1 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:20.211240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-d57782f0-e331-463a-9660-111fc2c1d7cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:20.211240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:20.211240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:20.211240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:20.211240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:20.211240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:20.211240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:20.211240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:20.211240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:20.211240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:20.211240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:20.211240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:20.211240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:20.211240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:20.211240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:20.211240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:20.212192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:20.212192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:20.212192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:20.212192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:20.212192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:20.212192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:20.212192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:20.212192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:20.212192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:20.212192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:20.212192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:20.212192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:20.212192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:20.212192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:20.212192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:20.212192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:20.213058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:20.213058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:20.213058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:20.213058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:20.213058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:20.213058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:20.213058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:20.213058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:20.213058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:20.213058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:20.213058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:20.213058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:20.213058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 293/571] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:56:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:20.213058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57782f0-e331-463a-9660-111fc2c1d7cd None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:20.258755 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:20.308654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74d174-642a-4855-b5c7-9ffb2b17c2b1 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:20.313134 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57782f0-e331-463a-9660-111fc2c1d7cd None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:20.341954 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabc3c1-a01f-4801-bb64-1ca0b74df239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:20.342587 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabc3c1-a01f-4801-bb64-1ca0b74df239 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:20.342870 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabc3c1-a01f-4801-bb64-1ca0b74df239 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:20.343173 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cabc3c1-a01f-4801-bb64-1ca0b74df239 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:20.345520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98060b-ca27-4305-b092-a36d2e019a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:20.346166 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98060b-ca27-4305-b092-a36d2e019a24 None admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:20.346444 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98060b-ca27-4305-b092-a36d2e019a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:20.346711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98060b-ca27-4305-b092-a36d2e019a24 None admin] PATH_INFO: `/v3/projects/eef80493db09437b88f2b147a59928c1/users/cce502a83f5343558c2136c0193e7d85/roles/637ed3cd3d484839ab21de02c2bbe293` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:20.354655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cabc3c1-a01f-4801-bb64-1ca0b74df239 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:20.359963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-7cabc3c1-a01f-4801-bb64-1ca0b74df239 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:20.359963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:20.359963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:20.359963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:20.359963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:20.359963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:20.359963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:20.359963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:20.359963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:20.359963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:20.359963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:20.359963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:20.359963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:20.359963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:20.359963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:20.359963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:20.360875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:20.360875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:20.360875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:20.360875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:20.360875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:20.360875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:20.360875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:20.360875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:20.360875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:20.360875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:20.360875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:20.360875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:20.360875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:20.360875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:20.360875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:20.360875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:20.361706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:20.361706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:20.361706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:20.361706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:20.361706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:20.361706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:20.361706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:20.361706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:20.361706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:20.361706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:20.361706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:20.361706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:20.361706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 279/572] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:56:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:20.362397 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabc3c1-a01f-4801-bb64-1ca0b74df239 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:20.364031 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98060b-ca27-4305-b092-a36d2e019a24 None admin] RBAC: Authorizing `identity:create_grant(project_id=eef80493db09437b88f2b147a59928c1, user_id=cce502a83f5343558c2136c0193e7d85, role_id=637ed3cd3d484839ab21de02c2bbe293)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:20.365480 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98060b-ca27-4305-b092-a36d2e019a24 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:20.377764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 294/573] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:56:20 2022] PUT /identity/v3/projects/eef80493db09437b88f2b147a59928c1/users/cce502a83f5343558c2136c0193e7d85/roles/637ed3cd3d484839ab21de02c2bbe293 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:56:20.377968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:20.378540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98060b-ca27-4305-b092-a36d2e019a24 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:20.482757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:20.486333 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cabc3c1-a01f-4801-bb64-1ca0b74df239 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:20.492612 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98060b-ca27-4305-b092-a36d2e019a24 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:20.528991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5843468-45ca-4458-8680-0e774bbbaca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:20.529624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5843468-45ca-4458-8680-0e774bbbaca9 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:20.529905 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5843468-45ca-4458-8680-0e774bbbaca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:20.530178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5843468-45ca-4458-8680-0e774bbbaca9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:20.536561 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b36a7-7349-4cea-acd0-6c1435b3318f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:20.537191 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b36a7-7349-4cea-acd0-6c1435b3318f None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:20.537465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b36a7-7349-4cea-acd0-6c1435b3318f None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:20.537732 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b36a7-7349-4cea-acd0-6c1435b3318f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:20.541945 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5843468-45ca-4458-8680-0e774bbbaca9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:20.543435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5843468-45ca-4458-8680-0e774bbbaca9 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:20.548152 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b36a7-7349-4cea-acd0-6c1435b3318f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:20.549577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b36a7-7349-4cea-acd0-6c1435b3318f None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:20.559128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-a5843468-45ca-4458-8680-0e774bbbaca9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:20.559128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:20.559128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:20.559128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:20.559128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:20.559128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:20.559128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:20.559128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:20.559128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:20.559128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:20.559128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:20.559128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:20.559128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:20.559128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:20.559128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:20.559128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:20.560652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:20.560652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:20.560652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:20.560652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:20.560652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:20.560652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:20.560652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:20.560652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:20.560652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:20.560652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:20.560652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:20.560652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:20.560652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:20.560652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:20.560652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:20.560652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:20.560652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:20.561527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:20.561527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:20.561527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:20.561527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 280/574] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:20.561761 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5843468-45ca-4458-8680-0e774bbbaca9 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:20.570224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-d45b36a7-7349-4cea-acd0-6c1435b3318f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:20.570224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:20.570224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:20.570224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:20.570224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:20.570224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:20.570224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:20.570224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:20.570224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:20.570224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:20.570224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:20.570224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:20.570224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:20.570224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:20.570224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:20.570224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:20.571232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:20.571232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:20.571232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:20.571232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:20.571232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:20.571232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:20.571232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:20.571232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:20.571232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:20.571232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:20.571232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:20.571232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:20.571232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:20.571232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:20.571232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:20.571232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:20.571232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:20.572329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:20.572329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:20.572329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:20.572329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 295/575] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:20.572948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b36a7-7349-4cea-acd0-6c1435b3318f None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:20.632745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:20.674704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5843468-45ca-4458-8680-0e774bbbaca9 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:20.675724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b36a7-7349-4cea-acd0-6c1435b3318f None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:20.710346 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e499d7-1046-4112-bfc4-ce96fa9f4d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:20.711091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54598ca-f757-4762-a0f7-b8a51d34e83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:20.711193 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e499d7-1046-4112-bfc4-ce96fa9f4d47 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:20.711268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e499d7-1046-4112-bfc4-ce96fa9f4d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:20.711512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54598ca-f757-4762-a0f7-b8a51d34e83b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:20.711624 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e499d7-1046-4112-bfc4-ce96fa9f4d47 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:20.711795 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54598ca-f757-4762-a0f7-b8a51d34e83b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:20.712062 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54598ca-f757-4762-a0f7-b8a51d34e83b None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:20.712220 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e499d7-1046-4112-bfc4-ce96fa9f4d47 None admin] RBAC: Adding query filter params (name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:20.712503 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e499d7-1046-4112-bfc4-ce96fa9f4d47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:20.712720 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54598ca-f757-4762-a0f7-b8a51d34e83b None admin] RBAC: Adding query filter params (name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:20.713023 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54598ca-f757-4762-a0f7-b8a51d34e83b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:20.713875 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e499d7-1046-4112-bfc4-ce96fa9f4d47 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:20.714522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54598ca-f757-4762-a0f7-b8a51d34e83b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:20.727187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 296/576] 10.222.0.36 () {56 vars in 1103 bytes} [Fri Sep 2 08:56:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:20.727187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e499d7-1046-4112-bfc4-ce96fa9f4d47 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:20.728345 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 281/577] 10.222.0.36 () {56 vars in 1103 bytes} [Fri Sep 2 08:56:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:20.729340 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54598ca-f757-4762-a0f7-b8a51d34e83b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:20.737647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:20.829853 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e499d7-1046-4112-bfc4-ce96fa9f4d47 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:20.843193 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54598ca-f757-4762-a0f7-b8a51d34e83b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:20.865258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d33e7a-665f-4f57-9df6-b69200d831a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:20.865862 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d33e7a-665f-4f57-9df6-b69200d831a0 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:20.866145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d33e7a-665f-4f57-9df6-b69200d831a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:20.866420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d33e7a-665f-4f57-9df6-b69200d831a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:20.867054 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d33e7a-665f-4f57-9df6-b69200d831a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:20.867384 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d33e7a-665f-4f57-9df6-b69200d831a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:20.868751 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d33e7a-665f-4f57-9df6-b69200d831a0 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:20.871260 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:20.880662 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 297/578] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:20.881064 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5be81d-d802-4192-bcb9-be2f17f4633b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:20.881458 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d33e7a-665f-4f57-9df6-b69200d831a0 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:20.882141 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5be81d-d802-4192-bcb9-be2f17f4633b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:20.882423 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5be81d-d802-4192-bcb9-be2f17f4633b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:20.882695 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5be81d-d802-4192-bcb9-be2f17f4633b None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:20.883380 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5be81d-d802-4192-bcb9-be2f17f4633b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:20.883675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5be81d-d802-4192-bcb9-be2f17f4633b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:20.885134 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5be81d-d802-4192-bcb9-be2f17f4633b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:20.899240 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 282/579] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:20.900223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5be81d-d802-4192-bcb9-be2f17f4633b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:20.958611 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:21.000025 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d33e7a-665f-4f57-9df6-b69200d831a0 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:21.004782 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5be81d-d802-4192-bcb9-be2f17f4633b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:21.037805 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e57fbb-e5e0-4659-92a8-d6fe4265cd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:21.038428 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e57fbb-e5e0-4659-92a8-d6fe4265cd74 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:21.038705 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e57fbb-e5e0-4659-92a8-d6fe4265cd74 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:21.038994 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e57fbb-e5e0-4659-92a8-d6fe4265cd74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:21.040131 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33433db0-668e-4129-a059-99499dbdc86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:21.040761 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33433db0-668e-4129-a059-99499dbdc86d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:21.041039 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33433db0-668e-4129-a059-99499dbdc86d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:21.041312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33433db0-668e-4129-a059-99499dbdc86d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:21.050399 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e57fbb-e5e0-4659-92a8-d6fe4265cd74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:21.051361 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33433db0-668e-4129-a059-99499dbdc86d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:21.051878 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e57fbb-e5e0-4659-92a8-d6fe4265cd74 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:21.052878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33433db0-668e-4129-a059-99499dbdc86d None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:21.067127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-d5e57fbb-e5e0-4659-92a8-d6fe4265cd74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:21.067127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:21.067127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:21.067127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:21.067127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:21.067127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:21.067127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:21.067127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:21.067127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:21.067127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:21.067127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:21.067127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:21.067127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:21.067127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:21.067127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:21.067127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:21.068092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:21.068092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:21.068092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:21.068092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:21.068092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:21.068092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:21.068092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:21.068092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:21.068092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:21.068092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:21.068092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:21.068092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:21.068092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:21.068092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:21.068092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:21.068092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:21.068092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:21.069076 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:21.069076 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:21.069076 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:21.069248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-33433db0-668e-4129-a059-99499dbdc86d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:21.069248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:21.069248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:21.069248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:21.069248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:21.069248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:21.069248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:21.069248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:21.069248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:21.069248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:21.069248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:21.069248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:21.069248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:21.069248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:21.069248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:21.069248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:21.070091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:21.070091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:21.070091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:21.070091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:21.070091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:21.070091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:21.070091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:21.070091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:21.070091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:21.070091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:21.070091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:21.070091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:21.070091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:21.070091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:21.070091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:21.070091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:21.070091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:21.072039 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:21.072039 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:21.072039 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:21.072039 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 283/580] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:21.072039 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33433db0-668e-4129-a059-99499dbdc86d None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:21.073003 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 298/581] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:21.073794 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e57fbb-e5e0-4659-92a8-d6fe4265cd74 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:21.115820 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:21.178426 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33433db0-668e-4129-a059-99499dbdc86d None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:21.179390 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e57fbb-e5e0-4659-92a8-d6fe4265cd74 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:21.202234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:21.215704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6468f37-4340-43fd-a3da-17d8d3cd6dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:21.216526 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6468f37-4340-43fd-a3da-17d8d3cd6dab None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:21.216947 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6468f37-4340-43fd-a3da-17d8d3cd6dab None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:21.217406 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6468f37-4340-43fd-a3da-17d8d3cd6dab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:21.218172 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76278bb5-5c3b-4a62-a226-922c24e2fbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:21.218802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76278bb5-5c3b-4a62-a226-922c24e2fbe8 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:21.219102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76278bb5-5c3b-4a62-a226-922c24e2fbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:21.219403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76278bb5-5c3b-4a62-a226-922c24e2fbe8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:21.224907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6468f37-4340-43fd-a3da-17d8d3cd6dab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:21.226536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6468f37-4340-43fd-a3da-17d8d3cd6dab None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:21.227331 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76278bb5-5c3b-4a62-a226-922c24e2fbe8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:21.228769 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76278bb5-5c3b-4a62-a226-922c24e2fbe8 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:21.238580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-f6468f37-4340-43fd-a3da-17d8d3cd6dab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:21.238580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:21.238580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:21.238580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:21.238580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:21.238580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:21.238580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:21.238580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:21.238580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:21.238580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:21.238580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:21.238580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:21.238580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:21.238580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:21.238580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:21.239541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:21.239541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:21.239541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:21.239541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:21.239541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:21.239541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:21.239541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:21.239541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:21.239541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:21.239541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:21.239541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:21.239541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:21.239541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:21.239541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:21.239541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:21.239541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:21.239541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:21.240460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:21.240460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:21.240460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:21.240460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:21.240460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:21.240460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:21.240460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:21.240460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:21.240914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-76278bb5-5c3b-4a62-a226-922c24e2fbe8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 02 08:56:21.240914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:21.240914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:21.240914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:21.240914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:21.240914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:21.240914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:21.240914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:21.240914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:21.240914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:21.240914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:21.240914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:21.240914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:21.240914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:21.240914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:21.240914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:21.242465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:21.242465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:21.242465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:21.242465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:21.242465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:21.242465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:21.242465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:21.242465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:21.242465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:21.242465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:21.242465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:21.242465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:21.242465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:21.242465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:21.242465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:21.242465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:21.242465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:21.242465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:21.243440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:21.243440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:21.243440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:21.243440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:21.243440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:21.243440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:21.243440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:21.243440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:21.243440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 02 08:56:21.243440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:21.243440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 299/582] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:56:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:21.243440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76278bb5-5c3b-4a62-a226-922c24e2fbe8 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:21.245734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 284/583] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:21.246555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6468f37-4340-43fd-a3da-17d8d3cd6dab None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:21.356823 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76278bb5-5c3b-4a62-a226-922c24e2fbe8 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:21.359608 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:21.364363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6468f37-4340-43fd-a3da-17d8d3cd6dab None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:21.395398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e585c9-d6e5-4793-b2bc-3050d065e9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:21.396002 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e585c9-d6e5-4793-b2bc-3050d065e9b0 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:21.396278 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e585c9-d6e5-4793-b2bc-3050d065e9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:21.396547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e585c9-d6e5-4793-b2bc-3050d065e9b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:21.397183 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e585c9-d6e5-4793-b2bc-3050d065e9b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:21.397482 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e585c9-d6e5-4793-b2bc-3050d065e9b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:21.398834 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e585c9-d6e5-4793-b2bc-3050d065e9b0 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:21.407223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0603d5-ea7f-4de6-84c5-df34653a3270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:21.407832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0603d5-ea7f-4de6-84c5-df34653a3270 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:21.408113 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0603d5-ea7f-4de6-84c5-df34653a3270 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:21.408382 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0603d5-ea7f-4de6-84c5-df34653a3270 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:21.409025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0603d5-ea7f-4de6-84c5-df34653a3270 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:21.409328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0603d5-ea7f-4de6-84c5-df34653a3270 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:21.410737 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0603d5-ea7f-4de6-84c5-df34653a3270 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:21.416146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 300/584] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:21.416146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e585c9-d6e5-4793-b2bc-3050d065e9b0 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:21.433013 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 285/585] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:21.433013 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0603d5-ea7f-4de6-84c5-df34653a3270 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:21.462138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:21.522774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e585c9-d6e5-4793-b2bc-3050d065e9b0 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:21.546382 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0603d5-ea7f-4de6-84c5-df34653a3270 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:21.556446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac6d8a-1f45-4b63-a813-81605c83301a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:21.557048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac6d8a-1f45-4b63-a813-81605c83301a None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:21.557373 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac6d8a-1f45-4b63-a813-81605c83301a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:21.557604 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bac6d8a-1f45-4b63-a813-81605c83301a None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:21.558417 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac6d8a-1f45-4b63-a813-81605c83301a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:21.558724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac6d8a-1f45-4b63-a813-81605c83301a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:21.560426 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bac6d8a-1f45-4b63-a813-81605c83301a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:21.578505 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 301/586] 10.222.0.36 () {56 vars in 1137 bytes} [Fri Sep 2 08:56:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:21.579412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac6d8a-1f45-4b63-a813-81605c83301a None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:21.591310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58900d-12c6-4363-9ee3-681d802a01b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:21.591917 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58900d-12c6-4363-9ee3-681d802a01b2 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:21.592466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58900d-12c6-4363-9ee3-681d802a01b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:21.592466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58900d-12c6-4363-9ee3-681d802a01b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:21.593172 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58900d-12c6-4363-9ee3-681d802a01b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:21.593483 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58900d-12c6-4363-9ee3-681d802a01b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:21.594934 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58900d-12c6-4363-9ee3-681d802a01b2 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:21.607830 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 286/587] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:21.612424 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58900d-12c6-4363-9ee3-681d802a01b2 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:21.632093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:21.683862 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:21.688278 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bac6d8a-1f45-4b63-a813-81605c83301a None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:21.716080 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58900d-12c6-4363-9ee3-681d802a01b2 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:21.725228 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600cffe-ec37-4027-8b8f-7a663b577d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:21.726765 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600cffe-ec37-4027-8b8f-7a663b577d55 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:21.727622 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600cffe-ec37-4027-8b8f-7a663b577d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:21.728245 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600cffe-ec37-4027-8b8f-7a663b577d55 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:21.737034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600cffe-ec37-4027-8b8f-7a663b577d55 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:21.739505 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600cffe-ec37-4027-8b8f-7a663b577d55 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:21.750951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c495d-af36-4bab-b3fd-f261b75cdb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:21.751708 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c495d-af36-4bab-b3fd-f261b75cdb93 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:21.752072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-1600cffe-ec37-4027-8b8f-7a663b577d55 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 02 08:56:21.752072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:21.752072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:21.752072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:21.752072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:21.752072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:21.752072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:21.752072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:21.752072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:21.752072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:21.752072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:21.752072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:21.752072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:21.752072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:21.752072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:21.752072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:21.753145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:21.753145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:21.753145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:21.753145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:21.753145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:21.753145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:21.753145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:21.753145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:21.753145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:21.753145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:21.753145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:21.753145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:21.753145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:21.753145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:21.753145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:21.753145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:21.753145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:21.753145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:21.754432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:21.754432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:21.754432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:21.754432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:21.754432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:21.754432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:21.754432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:21.754432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:21.754432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 02 08:56:21.754432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:21.755112 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c495d-af36-4bab-b3fd-f261b75cdb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:21.755112 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c495d-af36-4bab-b3fd-f261b75cdb93 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:21.758281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 302/588] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:56:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:21.758692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600cffe-ec37-4027-8b8f-7a663b577d55 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:21.763282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c495d-af36-4bab-b3fd-f261b75cdb93 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:21.764834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c495d-af36-4bab-b3fd-f261b75cdb93 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:21.776139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-1a3c495d-af36-4bab-b3fd-f261b75cdb93 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 08:56:21.776139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:21.776139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:21.776139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:21.776139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:21.776139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:21.776139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:21.776139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:21.776139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:21.776139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:21.776139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:21.776139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:21.776139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:21.776139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:21.776139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:21.776139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:21.777291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:21.777291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:21.777291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:21.777291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:21.777291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:21.777291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:21.777291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:21.777291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:21.777291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:21.777291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:21.777291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:21.777291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:21.777291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:21.777291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:21.777291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:21.777291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:21.777291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:21.777291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:21.778230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:21.778230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:21.778230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:21.778230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:21.778230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:21.778230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:21.778230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:21.778230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:21.778230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 08:56:21.778230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:21.778230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 287/589] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:56:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:21.778810 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c495d-af36-4bab-b3fd-f261b75cdb93 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:21.861416 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:21.871510 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600cffe-ec37-4027-8b8f-7a663b577d55 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:21.883968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c495d-af36-4bab-b3fd-f261b75cdb93 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:21.906196 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8d97d-ea40-4def-9b66-2436d8b6e2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:21.907102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8d97d-ea40-4def-9b66-2436d8b6e2e7 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:21.907519 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8d97d-ea40-4def-9b66-2436d8b6e2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:21.907930 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8d97d-ea40-4def-9b66-2436d8b6e2e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:21.919549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39f125-d811-4746-a395-ffbdeb50bd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:21.919810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8d97d-ea40-4def-9b66-2436d8b6e2e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:21.920160 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39f125-d811-4746-a395-ffbdeb50bd48 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:21.920433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39f125-d811-4746-a395-ffbdeb50bd48 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:21.920709 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39f125-d811-4746-a395-ffbdeb50bd48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:21.921438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39f125-d811-4746-a395-ffbdeb50bd48 None admin] RBAC: Adding query filter params (role.id=637ed3cd3d484839ab21de02c2bbe293, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=92318948dcf1407b9fc8aa5c199787ae) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:21.921745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39f125-d811-4746-a395-ffbdeb50bd48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:21.922193 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8d97d-ea40-4def-9b66-2436d8b6e2e7 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:21.923181 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba39f125-d811-4746-a395-ffbdeb50bd48 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:21.929078 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 288/590] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:56:21 2022] GET /identity/v3/role_assignments?user.id=92318948dcf1407b9fc8aa5c199787ae&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=637ed3cd3d484839ab21de02c2bbe293 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:21.929852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39f125-d811-4746-a395-ffbdeb50bd48 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:21.950669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-b9f8d97d-ea40-4def-9b66-2436d8b6e2e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:21.950669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:21.950669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:21.950669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:21.950669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:21.950669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:21.950669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:21.950669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:21.950669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:21.950669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:21.950669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:21.950669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:21.950669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:21.950669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:21.950669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:21.950669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:21.951741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:21.951741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:21.951741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:21.951741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:21.951741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:21.951741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:21.951741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:21.951741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:21.951741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:21.951741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:21.951741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:21.951741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:21.951741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:21.951741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:21.951741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:21.951741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:21.951741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:21.952640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:21.952640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:21.952640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:21.955462 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 303/591] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:21.957204 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8d97d-ea40-4def-9b66-2436d8b6e2e7 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:21.971609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:22.071582 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8d97d-ea40-4def-9b66-2436d8b6e2e7 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:22.085167 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39f125-d811-4746-a395-ffbdeb50bd48 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:22.110175 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cb27c-ffbb-4650-83a6-62c577e344e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:22.110970 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cb27c-ffbb-4650-83a6-62c577e344e8 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:22.111384 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cb27c-ffbb-4650-83a6-62c577e344e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:22.111839 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cb27c-ffbb-4650-83a6-62c577e344e8 None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:22.112738 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3cb27c-ffbb-4650-83a6-62c577e344e8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:22.113163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3cb27c-ffbb-4650-83a6-62c577e344e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:22.114675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3cb27c-ffbb-4650-83a6-62c577e344e8 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:22.126993 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9a30b-615c-4b87-962a-ebad0f04ba07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:22.127867 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9a30b-615c-4b87-962a-ebad0f04ba07 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:22.128295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9a30b-615c-4b87-962a-ebad0f04ba07 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:22.128754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc9a30b-615c-4b87-962a-ebad0f04ba07 None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:22.129682 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9a30b-615c-4b87-962a-ebad0f04ba07 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:22.130235 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9a30b-615c-4b87-962a-ebad0f04ba07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:22.131968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc9a30b-615c-4b87-962a-ebad0f04ba07 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:22.134336 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 304/592] 10.222.0.36 () {56 vars in 1137 bytes} [Fri Sep 2 08:56:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:22.135293 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cb27c-ffbb-4650-83a6-62c577e344e8 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:22.148385 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:22.157298 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 289/593] 10.222.0.36 () {56 vars in 1139 bytes} [Fri Sep 2 08:56:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:22.158504 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9a30b-615c-4b87-962a-ebad0f04ba07 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:22.208952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:22.246351 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cb27c-ffbb-4650-83a6-62c577e344e8 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:22.273946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc9a30b-615c-4b87-962a-ebad0f04ba07 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:22.283916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889ef04-0e73-4d36-b787-a2a6ff06edb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:22.284867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889ef04-0e73-4d36-b787-a2a6ff06edb4 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:22.285872 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889ef04-0e73-4d36-b787-a2a6ff06edb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:22.286349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889ef04-0e73-4d36-b787-a2a6ff06edb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:22.287228 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889ef04-0e73-4d36-b787-a2a6ff06edb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:22.287632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889ef04-0e73-4d36-b787-a2a6ff06edb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:22.289154 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889ef04-0e73-4d36-b787-a2a6ff06edb4 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:22.302871 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 305/594] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:22.304003 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889ef04-0e73-4d36-b787-a2a6ff06edb4 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:22.314341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64628d5a-f2a1-4825-bc11-1e4e56cace34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:22.315399 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64628d5a-f2a1-4825-bc11-1e4e56cace34 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:22.315907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64628d5a-f2a1-4825-bc11-1e4e56cace34 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:22.316540 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64628d5a-f2a1-4825-bc11-1e4e56cace34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:22.327601 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64628d5a-f2a1-4825-bc11-1e4e56cace34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:22.329258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64628d5a-f2a1-4825-bc11-1e4e56cace34 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:22.345140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-64628d5a-f2a1-4825-bc11-1e4e56cace34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:22.345140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:22.345140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:22.345140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:22.345140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:22.345140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:22.345140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:22.345140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:22.345140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:22.345140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:22.345140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:22.345140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:22.345140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:22.345140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:22.345140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:22.345140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:22.346100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:22.346100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:22.346100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:22.346100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:22.346100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:22.346100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:22.346100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:22.346100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:22.346100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:22.346100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:22.346100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:22.346100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:22.346100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:22.346100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:22.346100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:22.346100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:22.346100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:22.346977 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:22.346977 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:22.346977 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:22.346977 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 290/595] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:22.348596 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403915fb-bf66-4717-aa59-92ed6d776fba None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:22.348596 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403915fb-bf66-4717-aa59-92ed6d776fba None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:22.348758 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403915fb-bf66-4717-aa59-92ed6d776fba None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:22.354337 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 291/596] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:22.355150 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403915fb-bf66-4717-aa59-92ed6d776fba None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:22.413216 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:22.414254 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889ef04-0e73-4d36-b787-a2a6ff06edb4 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:22.442944 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:22.454576 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5e1c3-6eae-45a5-b8f5-200362bb245e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:22.455333 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5e1c3-6eae-45a5-b8f5-200362bb245e None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:22.455615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5e1c3-6eae-45a5-b8f5-200362bb245e None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:22.455890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5e1c3-6eae-45a5-b8f5-200362bb245e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:22.467311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5e1c3-6eae-45a5-b8f5-200362bb245e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:22.468759 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5e1c3-6eae-45a5-b8f5-200362bb245e None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:22.472108 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403915fb-bf66-4717-aa59-92ed6d776fba None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:22.483258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-b2e5e1c3-6eae-45a5-b8f5-200362bb245e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:22.483258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:22.483258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:22.483258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:22.483258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:22.483258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:22.483258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:22.483258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:22.483258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:22.483258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:22.483258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:22.483258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:22.483258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:22.483258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:22.483258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:22.483258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:22.484242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:22.484242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:22.484242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:22.484242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:22.484242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:22.484242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:22.484242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:22.484242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:22.484242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:22.484242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:22.484242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:22.484242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:22.484242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:22.484242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:22.484242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:22.484242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:22.484242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:22.485229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:22.485229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:22.485229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:22.485229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 306/597] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:22.485229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5e1c3-6eae-45a5-b8f5-200362bb245e None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:22.513240 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d535712-5d82-430d-8692-3fb915bac84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:22.514020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d535712-5d82-430d-8692-3fb915bac84f None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:22.514313 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d535712-5d82-430d-8692-3fb915bac84f None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:22.514592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d535712-5d82-430d-8692-3fb915bac84f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:22.522239 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d535712-5d82-430d-8692-3fb915bac84f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:22.523726 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d535712-5d82-430d-8692-3fb915bac84f None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:22.535103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-5d535712-5d82-430d-8692-3fb915bac84f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:22.535103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:22.535103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:22.535103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:22.535103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:22.535103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:22.535103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:22.535103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:22.535103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:22.535103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:22.535103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:22.535103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:22.535103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:22.535103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:22.535103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:22.535979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:22.535979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:22.535979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:22.535979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:22.535979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:22.535979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:22.535979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:22.535979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:22.535979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:22.535979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:22.535979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:22.535979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:22.535979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:22.535979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:22.535979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:22.535979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:22.535979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:22.536845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:22.536845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:22.536845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:22.536845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:22.536845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:22.536845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:22.536845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:22.536845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:22.536845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 292/598] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:22.537333 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af8bf2-fe77-4187-b724-b43fad08f448 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:22.537965 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af8bf2-fe77-4187-b724-b43fad08f448 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:22.537965 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af8bf2-fe77-4187-b724-b43fad08f448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:22.555337 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-98af8bf2-fe77-4187-b724-b43fad08f448 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:22.556537 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-98af8bf2-fe77-4187-b724-b43fad08f448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:22.598243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5e1c3-6eae-45a5-b8f5-200362bb245e None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:22.624741 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 293/599] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:22.625484 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af8bf2-fe77-4187-b724-b43fad08f448 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:22.639338 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81caacdf-6636-4093-8d9c-42d31251044a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:22.640222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81caacdf-6636-4093-8d9c-42d31251044a None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:22.640671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81caacdf-6636-4093-8d9c-42d31251044a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:22.641213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81caacdf-6636-4093-8d9c-42d31251044a None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:22.642014 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81caacdf-6636-4093-8d9c-42d31251044a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:22.642590 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81caacdf-6636-4093-8d9c-42d31251044a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:22.644810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81caacdf-6636-4093-8d9c-42d31251044a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:22.659159 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 307/600] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:22.660364 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81caacdf-6636-4093-8d9c-42d31251044a None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:22.692202 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:22.711886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:22.756873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98af8bf2-fe77-4187-b724-b43fad08f448 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:22.776513 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81caacdf-6636-4093-8d9c-42d31251044a None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:22.807323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989e538-56f6-4834-90ee-5eab4d734cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:22.807935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989e538-56f6-4834-90ee-5eab4d734cdf None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:22.808215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989e538-56f6-4834-90ee-5eab4d734cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:22.808487 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989e538-56f6-4834-90ee-5eab4d734cdf None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:22.809134 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989e538-56f6-4834-90ee-5eab4d734cdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:22.809689 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989e538-56f6-4834-90ee-5eab4d734cdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:22.811051 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989e538-56f6-4834-90ee-5eab4d734cdf None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:22.825520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0454d47-8220-478f-acdc-c7661b0bb9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:22.826475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0454d47-8220-478f-acdc-c7661b0bb9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:22.826959 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0454d47-8220-478f-acdc-c7661b0bb9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:22.827254 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 294/601] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:22.827688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0454d47-8220-478f-acdc-c7661b0bb9c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:22.828097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989e538-56f6-4834-90ee-5eab4d734cdf None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:22.828926 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0454d47-8220-478f-acdc-c7661b0bb9c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:22.829449 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0454d47-8220-478f-acdc-c7661b0bb9c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:22.832525 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0454d47-8220-478f-acdc-c7661b0bb9c2 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:22.849161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 308/602] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:22.850309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0454d47-8220-478f-acdc-c7661b0bb9c2 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:22.938394 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989e538-56f6-4834-90ee-5eab4d734cdf None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:22.940084 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:22.974017 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:22.974644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1bd80-e782-410c-be43-1fc41f83a24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:22.975314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1bd80-e782-410c-be43-1fc41f83a24d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:22.975598 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1bd80-e782-410c-be43-1fc41f83a24d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:22.975916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b1bd80-e782-410c-be43-1fc41f83a24d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:22.977790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0454d47-8220-478f-acdc-c7661b0bb9c2 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:22.984229 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1bd80-e782-410c-be43-1fc41f83a24d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:22.986180 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b1bd80-e782-410c-be43-1fc41f83a24d None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:22.999362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-26b1bd80-e782-410c-be43-1fc41f83a24d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:22.999362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:22.999362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:22.999362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:22.999362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:22.999362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:22.999362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:22.999362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:22.999362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:22.999362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:22.999362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:22.999362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:22.999362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:22.999362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:22.999362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:23.000270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:23.000270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:23.000270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:23.000270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:23.000270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:23.000270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:23.000270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:23.000270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:23.000270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:23.000270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:23.000270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:23.000270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:23.000270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:23.000270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:23.000270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:23.000270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:23.000270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:23.001185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:23.001185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:23.001185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:23.001185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:23.001185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:23.001185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:23.001185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:23.001185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:23.001185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 295/603] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:23.001673 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1bd80-e782-410c-be43-1fc41f83a24d None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:23.016261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545df01-d277-4bec-884f-c7509955f2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:23.016888 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545df01-d277-4bec-884f-c7509955f2f5 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:23.017165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545df01-d277-4bec-884f-c7509955f2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:23.017436 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545df01-d277-4bec-884f-c7509955f2f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:23.025809 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c545df01-d277-4bec-884f-c7509955f2f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:23.027736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c545df01-d277-4bec-884f-c7509955f2f5 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:23.042161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-c545df01-d277-4bec-884f-c7509955f2f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:23.042161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:23.042161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:23.042161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:23.042161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:23.042161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:23.042161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:23.042161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:23.042161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:23.042161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:23.042161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:23.042161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:23.042161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:23.042161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:23.042161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:23.043052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:23.043052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:23.043052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:23.043052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:23.043052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:23.043052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:23.043052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:23.043052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:23.043052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:23.043052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:23.043052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:23.043052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:23.043052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:23.043052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:23.043052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:23.043052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:23.043052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:23.044006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:23.044006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:23.044006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:23.044006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:23.044006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:23.044006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:23.044006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:23.044006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:23.045162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 309/604] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:23.045816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545df01-d277-4bec-884f-c7509955f2f5 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:23.119210 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b1bd80-e782-410c-be43-1fc41f83a24d None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:23.154318 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d442ba5-279a-4d7c-9dba-4a4c60f8cbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:23.154501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545df01-d277-4bec-884f-c7509955f2f5 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:23.154976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d442ba5-279a-4d7c-9dba-4a4c60f8cbc6 None admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:23.155285 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d442ba5-279a-4d7c-9dba-4a4c60f8cbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:23.155564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d442ba5-279a-4d7c-9dba-4a4c60f8cbc6 None admin] PATH_INFO: `/v3/projects/eef80493db09437b88f2b147a59928c1/users/2474d89f10764afbbc5e740dccaae4df/roles/637ed3cd3d484839ab21de02c2bbe293` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:23.171402 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d442ba5-279a-4d7c-9dba-4a4c60f8cbc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=eef80493db09437b88f2b147a59928c1, user_id=2474d89f10764afbbc5e740dccaae4df, role_id=637ed3cd3d484839ab21de02c2bbe293)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:23.172801 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d442ba5-279a-4d7c-9dba-4a4c60f8cbc6 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:23.187341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 296/605] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:56:23 2022] PUT /identity/v3/projects/eef80493db09437b88f2b147a59928c1/users/2474d89f10764afbbc5e740dccaae4df/roles/637ed3cd3d484839ab21de02c2bbe293 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:56:23.188520 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d442ba5-279a-4d7c-9dba-4a4c60f8cbc6 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:23.197494 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279e7a3-7325-4664-a039-2ad694030dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:23.198428 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279e7a3-7325-4664-a039-2ad694030dcb None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:23.198711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279e7a3-7325-4664-a039-2ad694030dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:23.199027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279e7a3-7325-4664-a039-2ad694030dcb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:23.212311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279e7a3-7325-4664-a039-2ad694030dcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:23.213682 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:23.219240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-e279e7a3-7325-4664-a039-2ad694030dcb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:23.219240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:23.219240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:23.219240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:23.219240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:23.219240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:23.219240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:23.219240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:23.219240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:23.219240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:23.219240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:23.219240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:23.219240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:23.219240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:23.219240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:23.219240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:23.220481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:23.220481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:23.220481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:23.220481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:23.220481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:23.220481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:23.220481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:23.220481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:23.220481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:23.220481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:23.220481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:23.220481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:23.220481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:23.220481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:23.220481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:23.220481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:23.222160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:23.222160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:23.222160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:23.222160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:23.222160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:23.222160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:23.222160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:23.222160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:23.222160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:23.222160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:23.222160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:23.222160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:23.222160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 310/606] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:56:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:23.227060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279e7a3-7325-4664-a039-2ad694030dcb None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:23.252526 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:23.343034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d442ba5-279a-4d7c-9dba-4a4c60f8cbc6 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:23.343657 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279e7a3-7325-4664-a039-2ad694030dcb None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:23.385024 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace9c6b-95ff-4bff-91df-b63a81615511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:23.385628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace9c6b-95ff-4bff-91df-b63a81615511 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:23.386140 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace9c6b-95ff-4bff-91df-b63a81615511 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:23.386140 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace9c6b-95ff-4bff-91df-b63a81615511 None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:23.386835 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace9c6b-95ff-4bff-91df-b63a81615511 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:23.387182 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace9c6b-95ff-4bff-91df-b63a81615511 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:23.388617 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace9c6b-95ff-4bff-91df-b63a81615511 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:23.394593 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141805ea-43f8-485a-832c-a241346da7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:23.395552 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141805ea-43f8-485a-832c-a241346da7b2 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:23.395973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141805ea-43f8-485a-832c-a241346da7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:23.396902 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141805ea-43f8-485a-832c-a241346da7b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:23.397744 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141805ea-43f8-485a-832c-a241346da7b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:23.398201 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141805ea-43f8-485a-832c-a241346da7b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:23.400572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141805ea-43f8-485a-832c-a241346da7b2 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:23.402053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 311/607] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:23.403276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace9c6b-95ff-4bff-91df-b63a81615511 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:23.416232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 297/608] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:23.417684 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141805ea-43f8-485a-832c-a241346da7b2 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:23.502724 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:23.513146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:23.537135 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace9c6b-95ff-4bff-91df-b63a81615511 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:23.546513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141805ea-43f8-485a-832c-a241346da7b2 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:23.574876 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29495851-b6af-4025-96fe-8b7dbfdacd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:23.575366 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29495851-b6af-4025-96fe-8b7dbfdacd06 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:23.575645 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29495851-b6af-4025-96fe-8b7dbfdacd06 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:23.575918 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29495851-b6af-4025-96fe-8b7dbfdacd06 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:23.576578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29495851-b6af-4025-96fe-8b7dbfdacd06 None admin] RBAC: Adding query filter params (name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:23.576879 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29495851-b6af-4025-96fe-8b7dbfdacd06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:23.578248 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29495851-b6af-4025-96fe-8b7dbfdacd06 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:23.587632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 312/609] 10.222.0.36 () {56 vars in 1103 bytes} [Fri Sep 2 08:56:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:23.588250 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29495851-b6af-4025-96fe-8b7dbfdacd06 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:23.594579 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574d77f-16d0-46d3-a00e-67762e702fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:23.596954 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574d77f-16d0-46d3-a00e-67762e702fd8 None admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:23.596954 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574d77f-16d0-46d3-a00e-67762e702fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:23.596954 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574d77f-16d0-46d3-a00e-67762e702fd8 None admin] PATH_INFO: `/v3/projects/eef80493db09437b88f2b147a59928c1/users/07a825fddbc04f918bcd631976038cf9/roles/637ed3cd3d484839ab21de02c2bbe293` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:23.616673 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574d77f-16d0-46d3-a00e-67762e702fd8 None admin] RBAC: Authorizing `identity:create_grant(project_id=eef80493db09437b88f2b147a59928c1, user_id=07a825fddbc04f918bcd631976038cf9, role_id=637ed3cd3d484839ab21de02c2bbe293)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:23.618563 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574d77f-16d0-46d3-a00e-67762e702fd8 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:23.638120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 298/610] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:56:23 2022] PUT /identity/v3/projects/eef80493db09437b88f2b147a59928c1/users/07a825fddbc04f918bcd631976038cf9/roles/637ed3cd3d484839ab21de02c2bbe293 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:56:23.639680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574d77f-16d0-46d3-a00e-67762e702fd8 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:23.773737 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29495851-b6af-4025-96fe-8b7dbfdacd06 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:23.828914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329f3f3-2159-4341-8eea-3bed703148d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:23.830375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329f3f3-2159-4341-8eea-3bed703148d8 None admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:23.830666 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329f3f3-2159-4341-8eea-3bed703148d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:23.830943 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329f3f3-2159-4341-8eea-3bed703148d8 None admin] PATH_INFO: `/v3/projects/eef80493db09437b88f2b147a59928c1/users/f8370135a78f414f9184c883ccb92767/roles/637ed3cd3d484839ab21de02c2bbe293` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:23.838845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574d77f-16d0-46d3-a00e-67762e702fd8 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:23.851164 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:23.867327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:23.868235 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b329f3f3-2159-4341-8eea-3bed703148d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=eef80493db09437b88f2b147a59928c1, user_id=f8370135a78f414f9184c883ccb92767, role_id=637ed3cd3d484839ab21de02c2bbe293)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:23.869799 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b329f3f3-2159-4341-8eea-3bed703148d8 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:23.879239 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af603b1-4fa8-49d8-a18c-a6a983c3166d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:23.879881 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af603b1-4fa8-49d8-a18c-a6a983c3166d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:23.880161 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af603b1-4fa8-49d8-a18c-a6a983c3166d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:23.880549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af603b1-4fa8-49d8-a18c-a6a983c3166d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:23.893061 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 313/611] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:56:23 2022] PUT /identity/v3/projects/eef80493db09437b88f2b147a59928c1/users/f8370135a78f414f9184c883ccb92767/roles/637ed3cd3d484839ab21de02c2bbe293 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:56:23.900767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af603b1-4fa8-49d8-a18c-a6a983c3166d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:23.902370 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af603b1-4fa8-49d8-a18c-a6a983c3166d None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:23.920878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-5af603b1-4fa8-49d8-a18c-a6a983c3166d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:23.920878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:23.920878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:23.920878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:23.920878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:23.920878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:23.920878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:23.920878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:23.920878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:23.920878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:23.920878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:23.920878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:23.920878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:23.920878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:23.920878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:23.920878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:23.921890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:23.921890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:23.921890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:23.921890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:23.921890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:23.921890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:23.921890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:23.921890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:23.921890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:23.921890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:23.921890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:23.921890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:23.921890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:23.921890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:23.921890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:23.921890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:23.921890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:23.923172 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:23.923172 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:23.923172 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:23.923172 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 299/612] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:23.928638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329f3f3-2159-4341-8eea-3bed703148d8 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:24.005180 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77232778-71e9-4c41-a91e-c2ca41f95a97 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:24.005180 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77232778-71e9-4c41-a91e-c2ca41f95a97 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:24.005180 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77232778-71e9-4c41-a91e-c2ca41f95a97 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:24.005180 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 300/613] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:24.006947 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4d1db-c238-4472-a7f0-c4ca4e0797f8 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:24.007341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4d1db-c238-4472-a7f0-c4ca4e0797f8 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:24.007683 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b4d1db-c238-4472-a7f0-c4ca4e0797f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:24.046760 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329f3f3-2159-4341-8eea-3bed703148d8 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:24.053314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-74b4d1db-c238-4472-a7f0-c4ca4e0797f8 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:24.054653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-74b4d1db-c238-4472-a7f0-c4ca4e0797f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:24.088394 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213209f-86f8-4fcd-820e-a8583e9da51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:24.089010 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213209f-86f8-4fcd-820e-a8583e9da51f None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:24.089288 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213209f-86f8-4fcd-820e-a8583e9da51f None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:24.089553 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213209f-86f8-4fcd-820e-a8583e9da51f None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:24.090190 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213209f-86f8-4fcd-820e-a8583e9da51f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:24.090486 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213209f-86f8-4fcd-820e-a8583e9da51f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:24.092190 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213209f-86f8-4fcd-820e-a8583e9da51f None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:24.105023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 314/614] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:24.110753 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213209f-86f8-4fcd-820e-a8583e9da51f None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:24.154044 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 301/615] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:56:24.187459 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:24.223738 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213209f-86f8-4fcd-820e-a8583e9da51f None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:24.276665 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c1507-0dd1-4afa-bd17-cb5ca19b36fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:24.277298 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c1507-0dd1-4afa-bd17-cb5ca19b36fc None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:24.277570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c1507-0dd1-4afa-bd17-cb5ca19b36fc None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:24.277834 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059c1507-0dd1-4afa-bd17-cb5ca19b36fc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:24.288067 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c1507-0dd1-4afa-bd17-cb5ca19b36fc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:24.289507 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059c1507-0dd1-4afa-bd17-cb5ca19b36fc None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:24.305091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-059c1507-0dd1-4afa-bd17-cb5ca19b36fc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 08:56:24.305091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:24.305091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:24.305091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:24.305091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:24.305091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:24.305091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:24.305091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:24.305091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:24.305091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:24.305091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:24.305091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:24.305091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:24.305091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:24.305091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:24.305091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:24.306034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:24.306034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:24.306034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:24.306034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:24.306034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:24.306034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:24.306034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:24.306034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:24.306034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:24.306034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:24.306034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:24.306034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:24.306034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:24.306034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:24.306034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:24.306034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:24.306034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:24.306034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:24.307294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:24.307294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:24.307294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:24.307294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:24.307294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:24.307294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:24.307294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:24.307294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:24.307294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 08:56:24.307294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:24.307294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 315/616] 10.222.0.36 () {56 vars in 1093 bytes} [Fri Sep 2 08:56:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:24.312768 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4d1db-c238-4472-a7f0-c4ca4e0797f8 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:24.456383 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c1507-0dd1-4afa-bd17-cb5ca19b36fc None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:24.467121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b4d1db-c238-4472-a7f0-c4ca4e0797f8 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:24.473934 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:24.532439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01aa34-bd36-4d20-8f23-fc2d2851b335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:24.533337 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01aa34-bd36-4d20-8f23-fc2d2851b335 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:24.533777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01aa34-bd36-4d20-8f23-fc2d2851b335 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:24.534193 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01aa34-bd36-4d20-8f23-fc2d2851b335 None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:24.535142 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01aa34-bd36-4d20-8f23-fc2d2851b335 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:24.535598 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01aa34-bd36-4d20-8f23-fc2d2851b335 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:24.537209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab01aa34-bd36-4d20-8f23-fc2d2851b335 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:24.565760 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 302/617] 10.222.0.36 () {56 vars in 1133 bytes} [Fri Sep 2 08:56:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:24.571693 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01aa34-bd36-4d20-8f23-fc2d2851b335 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:24.626313 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059c1507-0dd1-4afa-bd17-cb5ca19b36fc None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:24.646227 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:24.689301 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d23f8e-9da7-42fc-b2b3-85cbd8495e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:24.690182 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d23f8e-9da7-42fc-b2b3-85cbd8495e31 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:24.690612 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d23f8e-9da7-42fc-b2b3-85cbd8495e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:24.691601 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d23f8e-9da7-42fc-b2b3-85cbd8495e31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:24.713969 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d23f8e-9da7-42fc-b2b3-85cbd8495e31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:24.721560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-f4d23f8e-9da7-42fc-b2b3-85cbd8495e31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:24.721560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:24.721560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:24.721560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:24.721560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:24.721560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:24.721560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:24.721560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:24.721560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:24.721560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:24.721560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:24.721560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:24.721560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:24.721560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:24.721560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:24.721560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:24.723087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:24.723087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:24.723087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:24.723087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:24.723087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:24.723087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:24.723087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:24.723087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:24.723087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:24.723087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:24.723087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:24.723087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:24.723087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:24.723087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:24.723087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:24.723087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:24.723967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:24.723967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:24.723967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:24.723967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:24.723967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:24.723967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:24.723967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:24.723967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:24.723967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:24.723967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:24.723967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:24.723967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:24.726775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 316/618] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:56:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:24.734023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d23f8e-9da7-42fc-b2b3-85cbd8495e31 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:24.757571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab01aa34-bd36-4d20-8f23-fc2d2851b335 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:24.820426 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0fc06c-abde-45b3-b892-208b52163044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:24.820957 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fc06c-abde-45b3-b892-208b52163044 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:24.821235 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fc06c-abde-45b3-b892-208b52163044 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:24.821506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0fc06c-abde-45b3-b892-208b52163044 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:24.879432 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:24.919601 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d23f8e-9da7-42fc-b2b3-85cbd8495e31 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:24.944034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0fc06c-abde-45b3-b892-208b52163044 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:24.945501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0fc06c-abde-45b3-b892-208b52163044 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:24.982992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f26a3-4241-4a36-a15b-4d892773a0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:24.984736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f26a3-4241-4a36-a15b-4d892773a0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:24.984736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f26a3-4241-4a36-a15b-4d892773a0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:24.984736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329f26a3-4241-4a36-a15b-4d892773a0b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:24.985006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f26a3-4241-4a36-a15b-4d892773a0b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:24.985204 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f26a3-4241-4a36-a15b-4d892773a0b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:24.986537 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329f26a3-4241-4a36-a15b-4d892773a0b6 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:25.003739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 317/619] 10.222.0.36 () {56 vars in 1103 bytes} [Fri Sep 2 08:56:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:25.007250 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f26a3-4241-4a36-a15b-4d892773a0b6 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:25.075334 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:25.138711 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 303/620] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 02 08:56:25.142295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0fc06c-abde-45b3-b892-208b52163044 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:25.201824 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329f26a3-4241-4a36-a15b-4d892773a0b6 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:25.278098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:25.288366 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab5028-37c0-4ccf-b7d6-abd958bddc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:25.289017 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab5028-37c0-4ccf-b7d6-abd958bddc3c None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:25.289304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab5028-37c0-4ccf-b7d6-abd958bddc3c None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:25.289581 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab5028-37c0-4ccf-b7d6-abd958bddc3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:25.304899 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0fc06c-abde-45b3-b892-208b52163044 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:25.311063 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab5028-37c0-4ccf-b7d6-abd958bddc3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:25.312561 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab5028-37c0-4ccf-b7d6-abd958bddc3c None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:25.328659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-faab5028-37c0-4ccf-b7d6-abd958bddc3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:25.328659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:25.328659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:25.328659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:25.328659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:25.328659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:25.328659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:25.328659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:25.328659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:25.328659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:25.328659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:25.328659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:25.328659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:25.328659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:25.328659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:25.328659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:25.329659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:25.329659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:25.329659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:25.329659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:25.329659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:25.329659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:25.329659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:25.329659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:25.329659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:25.329659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:25.329659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:25.329659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:25.329659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:25.329659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:25.329659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:25.329659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:25.329659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:25.331200 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:25.331200 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:25.331200 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:25.331200 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 318/621] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:25.336265 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab5028-37c0-4ccf-b7d6-abd958bddc3c None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:25.373664 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27cc676-356c-48af-9a04-e41196eaa38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:25.374305 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cc676-356c-48af-9a04-e41196eaa38e None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:25.374580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cc676-356c-48af-9a04-e41196eaa38e None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:25.374847 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27cc676-356c-48af-9a04-e41196eaa38e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:25.385628 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27cc676-356c-48af-9a04-e41196eaa38e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:25.387099 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27cc676-356c-48af-9a04-e41196eaa38e None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:25.402437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-c27cc676-356c-48af-9a04-e41196eaa38e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:25.402437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:25.402437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:25.402437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:25.402437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:25.402437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:25.402437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:25.402437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:25.402437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:25.402437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:25.402437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:25.402437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:25.402437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:25.402437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:25.402437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:25.402437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:25.404675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:25.404675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:25.404675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:25.404675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:25.404675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:25.404675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:25.404675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:25.404675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:25.404675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:25.404675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:25.404675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:25.404675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:25.404675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:25.404675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:25.404675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:25.404675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:25.404675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:25.406783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:25.406783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:25.406783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:25.406783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 304/622] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:25.415093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27cc676-356c-48af-9a04-e41196eaa38e None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:25.432688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:25.450202 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab5028-37c0-4ccf-b7d6-abd958bddc3c None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:25.486609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2251e72-ff47-4942-9765-92df3fc11ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:25.487280 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2251e72-ff47-4942-9765-92df3fc11ebc None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:25.487611 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2251e72-ff47-4942-9765-92df3fc11ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:25.487922 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2251e72-ff47-4942-9765-92df3fc11ebc None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:25.488564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2251e72-ff47-4942-9765-92df3fc11ebc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:25.488868 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2251e72-ff47-4942-9765-92df3fc11ebc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:25.490358 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2251e72-ff47-4942-9765-92df3fc11ebc None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:25.504161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 319/623] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:25.505398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fc978-cf4d-41c5-9cbf-0f530c8f907e None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:25.505684 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fc978-cf4d-41c5-9cbf-0f530c8f907e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:25.505957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fc978-cf4d-41c5-9cbf-0f530c8f907e None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:25.511741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 320/624] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:25.512498 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fc978-cf4d-41c5-9cbf-0f530c8f907e None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:25.515106 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27cc676-356c-48af-9a04-e41196eaa38e None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:25.530418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:25.552269 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111090e2-f583-4324-9842-3a2f8c5e15ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:25.552943 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111090e2-f583-4324-9842-3a2f8c5e15ef None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:25.553222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111090e2-f583-4324-9842-3a2f8c5e15ef None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:25.553488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111090e2-f583-4324-9842-3a2f8c5e15ef None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:25.554123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111090e2-f583-4324-9842-3a2f8c5e15ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:25.554424 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111090e2-f583-4324-9842-3a2f8c5e15ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:25.555853 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111090e2-f583-4324-9842-3a2f8c5e15ef None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:25.569755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 305/625] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:25.570957 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156cdc4-81ef-41a3-bfa5-f1a4dfed92b7 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:25.575449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156cdc4-81ef-41a3-bfa5-f1a4dfed92b7 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:25.575748 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8156cdc4-81ef-41a3-bfa5-f1a4dfed92b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:25.594501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-8156cdc4-81ef-41a3-bfa5-f1a4dfed92b7 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:25.595714 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-8156cdc4-81ef-41a3-bfa5-f1a4dfed92b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:25.625355 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fc978-cf4d-41c5-9cbf-0f530c8f907e None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:25.662783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258cdcf6-c673-4eb2-8923-be09f6ee10de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:25.663701 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cdcf6-c673-4eb2-8923-be09f6ee10de None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:25.664079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cdcf6-c673-4eb2-8923-be09f6ee10de None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:25.664355 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cdcf6-c673-4eb2-8923-be09f6ee10de None admin] PATH_INFO: `/v3/users/swift` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:25.665382 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 306/626] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:25.666341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156cdc4-81ef-41a3-bfa5-f1a4dfed92b7 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:25.672283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258cdcf6-c673-4eb2-8923-be09f6ee10de None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:25.674027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258cdcf6-c673-4eb2-8923-be09f6ee10de None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:25.686727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-258cdcf6-c673-4eb2-8923-be09f6ee10de None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 02 08:56:25.686727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:25.686727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:25.686727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:25.686727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:25.686727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:25.686727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:25.686727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:25.686727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:25.686727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:25.686727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:25.686727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:25.686727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:25.686727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:25.686727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:25.686727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:25.689040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:25.689040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:25.689040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:25.689040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:25.689040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:25.689040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:25.689040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:25.689040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:25.689040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:25.689040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:25.689040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:25.689040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:25.689040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:25.689040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:25.689040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:25.689040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:25.689040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:25.689040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:25.691773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:25.691773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:25.691773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:25.691773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:25.691773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:25.691773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:25.691773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:25.691773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:25.691773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 02 08:56:25.691773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:25.691773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 321/627] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:56:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:25.699724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258cdcf6-c673-4eb2-8923-be09f6ee10de None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:25.703932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:25.770843 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8156cdc4-81ef-41a3-bfa5-f1a4dfed92b7 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:25.773330 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:25.807734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a5b6c-235c-4b25-9807-03073561ec05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:25.808366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a5b6c-235c-4b25-9807-03073561ec05 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:25.808641 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a5b6c-235c-4b25-9807-03073561ec05 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:25.808907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a5b6c-235c-4b25-9807-03073561ec05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:25.816853 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a5b6c-235c-4b25-9807-03073561ec05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:25.818341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150a5b6c-235c-4b25-9807-03073561ec05 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:25.819163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258cdcf6-c673-4eb2-8923-be09f6ee10de None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:25.829991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-150a5b6c-235c-4b25-9807-03073561ec05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:25.829991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:25.829991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:25.829991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:25.829991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:25.829991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:25.829991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:25.829991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:25.829991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:25.829991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:25.829991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:25.829991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:25.829991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:25.829991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:25.829991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:25.832154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:25.832154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:25.832154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:25.832154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:25.832154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:25.832154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:25.832154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:25.832154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:25.832154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:25.832154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:25.832154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:25.832154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:25.832154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:25.832154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:25.832154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:25.832154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:25.832154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:25.834274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:25.834274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:25.834274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:25.834274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:25.834274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:25.834274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:25.834274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:25.834274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:25.834274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 307/628] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:25.839109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a5b6c-235c-4b25-9807-03073561ec05 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:25.854806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e985d36-13f2-474f-b97e-6f9af9765204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:25.855449 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e985d36-13f2-474f-b97e-6f9af9765204 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:25.855730 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e985d36-13f2-474f-b97e-6f9af9765204 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:25.856006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e985d36-13f2-474f-b97e-6f9af9765204 None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:25.856744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e985d36-13f2-474f-b97e-6f9af9765204 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:25.857053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e985d36-13f2-474f-b97e-6f9af9765204 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:25.858488 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e985d36-13f2-474f-b97e-6f9af9765204 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:25.878341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 322/629] 10.222.0.36 () {56 vars in 1135 bytes} [Fri Sep 2 08:56:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:25.884178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e985d36-13f2-474f-b97e-6f9af9765204 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:25.943340 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a5b6c-235c-4b25-9807-03073561ec05 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:25.950050 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:25.991671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e985d36-13f2-474f-b97e-6f9af9765204 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:26.005955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee65324-73e6-44da-8dde-8e590ef31e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:26.006571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee65324-73e6-44da-8dde-8e590ef31e7d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:26.006849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee65324-73e6-44da-8dde-8e590ef31e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:26.007146 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee65324-73e6-44da-8dde-8e590ef31e7d None admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:26.007841 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee65324-73e6-44da-8dde-8e590ef31e7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:26.008143 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee65324-73e6-44da-8dde-8e590ef31e7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:26.009621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee65324-73e6-44da-8dde-8e590ef31e7d None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:26.021838 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 308/630] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:26.027639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee65324-73e6-44da-8dde-8e590ef31e7d None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:26.028512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e40b9-144a-4daa-8676-8c29cf0f84ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:26.029145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e40b9-144a-4daa-8676-8c29cf0f84ff None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:26.029425 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e40b9-144a-4daa-8676-8c29cf0f84ff None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:26.029694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e40b9-144a-4daa-8676-8c29cf0f84ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:26.040585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e40b9-144a-4daa-8676-8c29cf0f84ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:26.042041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e40b9-144a-4daa-8676-8c29cf0f84ff None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:26.054523 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:26.058035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-da8e40b9-144a-4daa-8676-8c29cf0f84ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:26.058035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:26.058035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:26.058035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:26.058035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:26.058035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:26.058035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:26.058035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:26.058035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:26.058035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:26.058035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:26.058035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:26.058035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:26.058035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:26.058035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:26.058035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:26.058966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:26.058966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:26.058966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:26.058966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:26.058966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:26.058966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:26.058966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:26.058966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:26.058966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:26.058966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:26.058966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:26.058966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:26.058966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:26.058966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:26.058966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:26.058966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:26.058966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:26.060323 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:26.060323 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:26.060323 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:26.060323 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 323/631] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:26.065539 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e40b9-144a-4daa-8676-8c29cf0f84ff None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:26.136316 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee65324-73e6-44da-8dde-8e590ef31e7d None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:26.170149 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d2ed0-5d03-4c8c-8427-8a2f6607cdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:26.171090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d2ed0-5d03-4c8c-8427-8a2f6607cdc6 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:26.171434 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d2ed0-5d03-4c8c-8427-8a2f6607cdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:26.171729 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d2ed0-5d03-4c8c-8427-8a2f6607cdc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:26.172610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d2ed0-5d03-4c8c-8427-8a2f6607cdc6 None admin] RBAC: Adding query filter params (role.id=637ed3cd3d484839ab21de02c2bbe293, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=cce502a83f5343558c2136c0193e7d85) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:26.173241 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d2ed0-5d03-4c8c-8427-8a2f6607cdc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:26.175104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d2ed0-5d03-4c8c-8427-8a2f6607cdc6 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:26.182334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 309/632] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:56:26 2022] GET /identity/v3/role_assignments?user.id=cce502a83f5343558c2136c0193e7d85&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=637ed3cd3d484839ab21de02c2bbe293 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:26.183127 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d2ed0-5d03-4c8c-8427-8a2f6607cdc6 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:26.183668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e40b9-144a-4daa-8676-8c29cf0f84ff None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:26.212761 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:26.244526 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb99f7-d423-4a5d-a83e-a0e78167d484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:26.245567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb99f7-d423-4a5d-a83e-a0e78167d484 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:26.246515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb99f7-d423-4a5d-a83e-a0e78167d484 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:26.247023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb99f7-d423-4a5d-a83e-a0e78167d484 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:26.248372 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb99f7-d423-4a5d-a83e-a0e78167d484 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:26.249150 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb99f7-d423-4a5d-a83e-a0e78167d484 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:26.251120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb99f7-d423-4a5d-a83e-a0e78167d484 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:26.270759 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 324/633] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:26.272543 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6d4a2-707e-4728-bad5-afc2385a8df7 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:26.273032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6d4a2-707e-4728-bad5-afc2385a8df7 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:26.273465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a6d4a2-707e-4728-bad5-afc2385a8df7 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:26.279093 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 325/634] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:26.280232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6d4a2-707e-4728-bad5-afc2385a8df7 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:26.312553 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:26.326756 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d2ed0-5d03-4c8c-8427-8a2f6607cdc6 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:26.370816 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af364392-abe6-46f9-bf9e-8818cdd592e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:26.371543 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af364392-abe6-46f9-bf9e-8818cdd592e4 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:26.371825 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af364392-abe6-46f9-bf9e-8818cdd592e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:26.372098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af364392-abe6-46f9-bf9e-8818cdd592e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:26.384079 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af364392-abe6-46f9-bf9e-8818cdd592e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:26.390955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-af364392-abe6-46f9-bf9e-8818cdd592e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:26.390955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:26.390955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:26.390955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:26.390955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:26.390955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:26.390955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:26.390955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:26.390955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:26.390955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:26.390955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:26.390955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:26.390955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:26.390955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:26.390955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:26.390955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:26.391930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:26.391930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:26.391930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:26.391930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:26.391930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:26.391930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:26.391930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:26.391930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:26.391930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:26.391930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:26.391930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:26.391930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:26.391930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:26.391930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:26.391930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:26.391930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:26.392766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:26.392766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:26.392766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:26.392766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:26.392766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:26.392766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:26.392766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:26.392766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:26.392766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:26.392766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:26.392766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:26.392766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:26.392766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 310/635] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:56:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:26.394727 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d444e-c519-462b-86c1-ea7e3137a1a9 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:26.394904 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d444e-c519-462b-86c1-ea7e3137a1a9 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:26.395850 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3d444e-c519-462b-86c1-ea7e3137a1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:26.416010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-3d3d444e-c519-462b-86c1-ea7e3137a1a9 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:26.417190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-3d3d444e-c519-462b-86c1-ea7e3137a1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:26.422770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a6d4a2-707e-4728-bad5-afc2385a8df7 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:26.462912 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1e50b-d229-4c99-839e-6438f9ef0749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:26.463835 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1e50b-d229-4c99-839e-6438f9ef0749 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:26.464113 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1e50b-d229-4c99-839e-6438f9ef0749 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:26.464383 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e1e50b-d229-4c99-839e-6438f9ef0749 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:26.473568 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1e50b-d229-4c99-839e-6438f9ef0749 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:26.475505 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e1e50b-d229-4c99-839e-6438f9ef0749 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:26.485165 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 311/636] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:26.486659 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cf125-108c-4a34-b6e8-d90d875b9213 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:26.487210 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cf125-108c-4a34-b6e8-d90d875b9213 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:26.487749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-48e1e50b-d229-4c99-839e-6438f9ef0749 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:26.487749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:26.487749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:26.487749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:26.487749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:26.487749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:26.487749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:26.487749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:26.487749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:26.487749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:26.487749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:26.487749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:26.487749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:26.487749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:26.487749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:26.489201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:26.489201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:26.489201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:26.489201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:26.489201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:26.489201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:26.489201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:26.489201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:26.489201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:26.489201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:26.489201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:26.489201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:26.489201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:26.489201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:26.489201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:26.489201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:26.489201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:26.490106 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:26.490106 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:26.490106 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:26.490106 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:26.490106 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:26.490106 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:26.490106 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:26.490106 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:26.490106 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 326/637] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:26.490604 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1e50b-d229-4c99-839e-6438f9ef0749 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:26.491182 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cf125-108c-4a34-b6e8-d90d875b9213 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:26.496986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 312/638] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:26 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:26.498075 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3cf125-108c-4a34-b6e8-d90d875b9213 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:26.526618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:26.588116 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:26.604690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e1e50b-d229-4c99-839e-6438f9ef0749 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:26.618771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3cf125-108c-4a34-b6e8-d90d875b9213 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:26.643118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f205fa1e-5538-4f57-87e9-f676fbd33b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:26.643641 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205fa1e-5538-4f57-87e9-f676fbd33b7c None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:26.644430 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205fa1e-5538-4f57-87e9-f676fbd33b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:26.644700 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f205fa1e-5538-4f57-87e9-f676fbd33b7c None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:26.645450 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205fa1e-5538-4f57-87e9-f676fbd33b7c None admin] RBAC: Adding query filter params (name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:26.645750 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205fa1e-5538-4f57-87e9-f676fbd33b7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:26.647163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f205fa1e-5538-4f57-87e9-f676fbd33b7c None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:26.657956 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 327/639] 10.222.0.36 () {56 vars in 1103 bytes} [Fri Sep 2 08:56:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:26.659229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf4f6c-17bc-4061-b467-89ba2a27a235 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:26.659627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf4f6c-17bc-4061-b467-89ba2a27a235 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:26.660951 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf4f6c-17bc-4061-b467-89ba2a27a235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:26.664757 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e9f07-f4a9-496b-8716-d3e16fe6fd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:26.665541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e9f07-f4a9-496b-8716-d3e16fe6fd94 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:26.665920 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e9f07-f4a9-496b-8716-d3e16fe6fd94 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:26.666284 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e9f07-f4a9-496b-8716-d3e16fe6fd94 None admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:26.667381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e9f07-f4a9-496b-8716-d3e16fe6fd94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:26.667802 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e9f07-f4a9-496b-8716-d3e16fe6fd94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:26.670181 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e9f07-f4a9-496b-8716-d3e16fe6fd94 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:26.681198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-baaf4f6c-17bc-4061-b467-89ba2a27a235 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:26.682768 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-baaf4f6c-17bc-4061-b467-89ba2a27a235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:26.683918 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 313/640] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:26.685449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e9f07-f4a9-496b-8716-d3e16fe6fd94 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:26.769983 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 328/641] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:26.771155 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf4f6c-17bc-4061-b467-89ba2a27a235 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:26.811315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:26.824099 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e9f07-f4a9-496b-8716-d3e16fe6fd94 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:26.858979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f941ea-2fbb-4dc4-b1d1-71d976d44bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:26.859642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f941ea-2fbb-4dc4-b1d1-71d976d44bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:26.859919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f941ea-2fbb-4dc4-b1d1-71d976d44bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:26.860186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f941ea-2fbb-4dc4-b1d1-71d976d44bf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:26.867546 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:26.872094 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f941ea-2fbb-4dc4-b1d1-71d976d44bf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:26.873644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f941ea-2fbb-4dc4-b1d1-71d976d44bf2 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:26.888314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-a1f941ea-2fbb-4dc4-b1d1-71d976d44bf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:26.888314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:26.888314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:26.888314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:26.888314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:26.888314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:26.888314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:26.888314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:26.888314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:26.888314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:26.888314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:26.888314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:26.888314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:26.888314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:26.888314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:26.888314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:26.889289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:26.889289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:26.889289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:26.889289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:26.889289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:26.889289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:26.889289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:26.889289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:26.889289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:26.889289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:26.889289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:26.889289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:26.889289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:26.889289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:26.889289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:26.889289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:26.889289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:26.890168 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:26.890168 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:26.890168 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:26.890168 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 314/642] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:26.890168 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f941ea-2fbb-4dc4-b1d1-71d976d44bf2 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:26.897438 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf4f6c-17bc-4061-b467-89ba2a27a235 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:26.934963 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282278e8-7ebc-49fc-8778-f34de044ef29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:26.935754 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282278e8-7ebc-49fc-8778-f34de044ef29 None admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:26.936041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282278e8-7ebc-49fc-8778-f34de044ef29 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:26.936412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282278e8-7ebc-49fc-8778-f34de044ef29 None admin] PATH_INFO: `/v3/projects/eef80493db09437b88f2b147a59928c1/users/92318948dcf1407b9fc8aa5c199787ae/roles/637ed3cd3d484839ab21de02c2bbe293` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:26.952687 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282278e8-7ebc-49fc-8778-f34de044ef29 None admin] RBAC: Authorizing `identity:create_grant(project_id=eef80493db09437b88f2b147a59928c1, user_id=92318948dcf1407b9fc8aa5c199787ae, role_id=637ed3cd3d484839ab21de02c2bbe293)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:26.954211 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282278e8-7ebc-49fc-8778-f34de044ef29 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:26.973178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 329/643] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:56:26 2022] PUT /identity/v3/projects/eef80493db09437b88f2b147a59928c1/users/92318948dcf1407b9fc8aa5c199787ae/roles/637ed3cd3d484839ab21de02c2bbe293 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:56:26.974345 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282278e8-7ebc-49fc-8778-f34de044ef29 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:26.994939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f941ea-2fbb-4dc4-b1d1-71d976d44bf2 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:27.033027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a69873-b219-4812-b54c-0d84ecc2b286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:27.033657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a69873-b219-4812-b54c-0d84ecc2b286 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:27.033931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a69873-b219-4812-b54c-0d84ecc2b286 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:27.034201 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a69873-b219-4812-b54c-0d84ecc2b286 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:27.046389 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a69873-b219-4812-b54c-0d84ecc2b286 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:27.051897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-60a69873-b219-4812-b54c-0d84ecc2b286 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:27.051897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:27.051897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:27.051897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:27.051897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:27.051897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:27.051897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:27.051897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:27.051897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:27.051897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:27.051897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:27.051897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:27.051897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:27.051897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:27.051897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:27.051897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:27.053789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:27.053789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:27.053789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:27.053789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:27.053789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:27.053789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:27.053789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:27.053789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:27.053789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:27.053789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:27.053789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:27.053789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:27.053789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:27.053789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:27.053789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:27.053789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:27.054915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:27.054915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:27.054915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:27.054915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:27.054915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:27.054915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:27.054915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:27.054915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:27.054915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:27.054915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:27.054915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:27.054915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:27.054915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 315/644] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:56:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:27.068287 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a69873-b219-4812-b54c-0d84ecc2b286 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:27.075889 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:27.101021 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282278e8-7ebc-49fc-8778-f34de044ef29 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:27.138456 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b83308-3b5a-4f3b-95e8-aeff5d5d89ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:27.139107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b83308-3b5a-4f3b-95e8-aeff5d5d89ea None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:27.139388 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b83308-3b5a-4f3b-95e8-aeff5d5d89ea None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:27.139666 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b83308-3b5a-4f3b-95e8-aeff5d5d89ea None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:27.140321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b83308-3b5a-4f3b-95e8-aeff5d5d89ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:27.140627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b83308-3b5a-4f3b-95e8-aeff5d5d89ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:27.142036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b83308-3b5a-4f3b-95e8-aeff5d5d89ea None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:27.159973 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 330/645] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:27.159973 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b83308-3b5a-4f3b-95e8-aeff5d5d89ea None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:27.163480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:27.231372 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a69873-b219-4812-b54c-0d84ecc2b286 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:27.268980 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b83308-3b5a-4f3b-95e8-aeff5d5d89ea None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:27.276393 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a42b4c-f473-4d35-b6ee-9e4e8fa45a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:27.277474 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a42b4c-f473-4d35-b6ee-9e4e8fa45a52 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:27.277755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a42b4c-f473-4d35-b6ee-9e4e8fa45a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:27.278032 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a42b4c-f473-4d35-b6ee-9e4e8fa45a52 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:27.279098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a42b4c-f473-4d35-b6ee-9e4e8fa45a52 None admin] RBAC: Adding query filter params (name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:27.279413 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a42b4c-f473-4d35-b6ee-9e4e8fa45a52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:27.281405 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a42b4c-f473-4d35-b6ee-9e4e8fa45a52 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:27.294672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 316/646] 10.222.0.36 () {56 vars in 1103 bytes} [Fri Sep 2 08:56:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:27.297685 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a42b4c-f473-4d35-b6ee-9e4e8fa45a52 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:27.317700 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6220789-a377-4501-99ab-9db99650b62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:27.318688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6220789-a377-4501-99ab-9db99650b62c None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:27.318971 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6220789-a377-4501-99ab-9db99650b62c None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:27.319688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6220789-a377-4501-99ab-9db99650b62c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:27.330135 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:27.337447 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6220789-a377-4501-99ab-9db99650b62c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:27.346783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-f6220789-a377-4501-99ab-9db99650b62c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:27.346783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:27.346783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:27.346783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:27.346783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:27.346783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:27.346783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:27.346783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:27.346783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:27.346783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:27.346783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:27.346783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:27.346783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:27.346783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:27.346783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:27.346783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:27.348032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:27.348032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:27.348032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:27.348032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:27.348032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:27.348032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:27.348032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:27.348032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:27.348032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:27.348032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:27.348032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:27.348032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:27.348032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:27.348032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:27.348032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:27.348032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:27.349132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:27.349132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:27.349132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:27.349132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:27.349132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:27.349132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:27.349132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:27.349132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:27.349132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:27.349132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:27.349132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:27.349132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:27.349132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 331/647] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:56:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:27.350303 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6220789-a377-4501-99ab-9db99650b62c None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:27.423352 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a42b4c-f473-4d35-b6ee-9e4e8fa45a52 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:27.445259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:27.471047 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6220789-a377-4501-99ab-9db99650b62c None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:27.473258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c4f24-8980-479a-8051-97a3c200cefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:27.473903 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c4f24-8980-479a-8051-97a3c200cefc None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:27.475011 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c4f24-8980-479a-8051-97a3c200cefc None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:27.475323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7c4f24-8980-479a-8051-97a3c200cefc None admin] PATH_INFO: `/v3/users/glance` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:27.484652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c4f24-8980-479a-8051-97a3c200cefc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:27.486272 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7c4f24-8980-479a-8051-97a3c200cefc None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:27.501582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-8e7c4f24-8980-479a-8051-97a3c200cefc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 02 08:56:27.501582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:27.501582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:27.501582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:27.501582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:27.501582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:27.501582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:27.501582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:27.501582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:27.501582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:27.501582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:27.501582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:27.501582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:27.501582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:27.501582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:27.501582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:27.502542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:27.502542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:27.502542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:27.502542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:27.502542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:27.502542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:27.502542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:27.502542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:27.502542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:27.502542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:27.502542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:27.502542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:27.502542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:27.502542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:27.502542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:27.502542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:27.502542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:27.502542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:27.504730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:27.504730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:27.504730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:27.504730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:27.504730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:27.504730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:27.504730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:27.504730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:27.504730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 02 08:56:27.504730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:27.504730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 317/648] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:56:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:27.504730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c4f24-8980-479a-8051-97a3c200cefc None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:27.511765 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d193f6f-813b-4746-b17c-bbfab3f226a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:27.512669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d193f6f-813b-4746-b17c-bbfab3f226a5 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:27.513234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d193f6f-813b-4746-b17c-bbfab3f226a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:27.513715 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d193f6f-813b-4746-b17c-bbfab3f226a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:27.526039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d193f6f-813b-4746-b17c-bbfab3f226a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:27.527774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d193f6f-813b-4746-b17c-bbfab3f226a5 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:27.543224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-6d193f6f-813b-4746-b17c-bbfab3f226a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:27.543224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:27.543224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:27.543224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:27.543224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:27.543224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:27.543224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:27.543224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:27.543224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:27.543224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:27.543224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:27.543224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:27.543224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:27.543224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:27.543224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:27.543224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:27.544162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:27.544162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:27.544162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:27.544162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:27.544162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:27.544162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:27.544162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:27.544162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:27.544162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:27.544162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:27.544162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:27.544162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:27.544162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:27.544162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:27.544162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:27.544162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:27.544162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:27.545074 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:27.545074 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:27.545074 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:27.547219 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 332/649] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:27.548283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d193f6f-813b-4746-b17c-bbfab3f226a5 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:27.608883 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:27.641195 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7c4f24-8980-479a-8051-97a3c200cefc None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:27.663321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d193f6f-813b-4746-b17c-bbfab3f226a5 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:27.688021 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451aa55-826b-453e-aa2a-d52b8a4f6cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:27.688974 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451aa55-826b-453e-aa2a-d52b8a4f6cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:27.689265 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451aa55-826b-453e-aa2a-d52b8a4f6cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:27.689543 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451aa55-826b-453e-aa2a-d52b8a4f6cf4 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:27.690491 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451aa55-826b-453e-aa2a-d52b8a4f6cf4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:27.691368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451aa55-826b-453e-aa2a-d52b8a4f6cf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:27.693078 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451aa55-826b-453e-aa2a-d52b8a4f6cf4 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:27.701487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff2f75-c01e-4393-b5f9-0e1f4c104d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:27.702363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff2f75-c01e-4393-b5f9-0e1f4c104d0c None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:27.702835 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff2f75-c01e-4393-b5f9-0e1f4c104d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:27.703338 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff2f75-c01e-4393-b5f9-0e1f4c104d0c None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:27.703570 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 318/650] 10.222.0.36 () {56 vars in 1103 bytes} [Fri Sep 2 08:56:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:27.704375 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451aa55-826b-453e-aa2a-d52b8a4f6cf4 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:27.704757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff2f75-c01e-4393-b5f9-0e1f4c104d0c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:27.705249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff2f75-c01e-4393-b5f9-0e1f4c104d0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:27.706954 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff2f75-c01e-4393-b5f9-0e1f4c104d0c None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:27.728999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 333/651] 10.222.0.36 () {56 vars in 1137 bytes} [Fri Sep 2 08:56:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:27.730232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff2f75-c01e-4393-b5f9-0e1f4c104d0c None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:27.748886 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:27.818315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451aa55-826b-453e-aa2a-d52b8a4f6cf4 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:27.859362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a6877-d5fb-4cfe-8515-c611d873d81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:27.859990 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a6877-d5fb-4cfe-8515-c611d873d81a None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:27.860249 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a6877-d5fb-4cfe-8515-c611d873d81a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:27.860528 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a6877-d5fb-4cfe-8515-c611d873d81a None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:27.861173 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a6877-d5fb-4cfe-8515-c611d873d81a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:27.861480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a6877-d5fb-4cfe-8515-c611d873d81a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:27.862354 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff2f75-c01e-4393-b5f9-0e1f4c104d0c None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:27.863437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9a6877-d5fb-4cfe-8515-c611d873d81a None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:27.873463 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:27.883148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 319/652] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:27.884396 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a6877-d5fb-4cfe-8515-c611d873d81a None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:27.905736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5319c08-9e5e-4471-b7fb-ec8551f45feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:27.906721 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5319c08-9e5e-4471-b7fb-ec8551f45feb None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:27.907333 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5319c08-9e5e-4471-b7fb-ec8551f45feb None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:27.907799 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5319c08-9e5e-4471-b7fb-ec8551f45feb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:27.919159 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5319c08-9e5e-4471-b7fb-ec8551f45feb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:27.920968 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5319c08-9e5e-4471-b7fb-ec8551f45feb None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:27.937744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-c5319c08-9e5e-4471-b7fb-ec8551f45feb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:27.937744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:27.937744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:27.937744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:27.937744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:27.937744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:27.937744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:27.937744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:27.937744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:27.937744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:27.937744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:27.937744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:27.937744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:27.937744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:27.937744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:27.937744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:27.938783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:27.938783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:27.938783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:27.938783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:27.938783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:27.938783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:27.938783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:27.938783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:27.938783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:27.938783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:27.938783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:27.938783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:27.938783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:27.938783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:27.938783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:27.938783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:27.938783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:27.939727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:27.939727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:27.939727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:27.941955 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 334/653] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:27.942996 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5319c08-9e5e-4471-b7fb-ec8551f45feb None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:28.007109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:28.016167 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a6877-d5fb-4cfe-8515-c611d873d81a None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:28.054632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5319c08-9e5e-4471-b7fb-ec8551f45feb None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:28.060901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d39b4c-b9a7-40a6-9dc9-828f20a0d27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:28.061821 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d39b4c-b9a7-40a6-9dc9-828f20a0d27a None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:28.062296 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d39b4c-b9a7-40a6-9dc9-828f20a0d27a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:28.062781 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d39b4c-b9a7-40a6-9dc9-828f20a0d27a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:28.091392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd96ecd-f251-433e-853f-59e8d13d4170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:28.092021 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd96ecd-f251-433e-853f-59e8d13d4170 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:28.092337 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd96ecd-f251-433e-853f-59e8d13d4170 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:28.092741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd96ecd-f251-433e-853f-59e8d13d4170 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:28.102233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd96ecd-f251-433e-853f-59e8d13d4170 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:28.103857 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd96ecd-f251-433e-853f-59e8d13d4170 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:28.116566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-9bd96ecd-f251-433e-853f-59e8d13d4170 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 02 08:56:28.116566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:28.116566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:28.116566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:28.116566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:28.116566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:28.116566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:28.116566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:28.116566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:28.116566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:28.116566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:28.116566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:28.116566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:28.116566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:28.116566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:28.116566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:28.117545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:28.117545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:28.117545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:28.117545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:28.117545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:28.117545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:28.117545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:28.117545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:28.117545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:28.117545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:28.117545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:28.117545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:28.117545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:28.117545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:28.117545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:28.117545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:28.117545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:28.117545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:28.118469 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:28.118469 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:28.118469 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:28.118469 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:28.118469 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:28.118469 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:28.118469 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:28.118469 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:28.118469 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 02 08:56:28.118469 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:28.122484 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 335/654] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:56:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:28.123338 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd96ecd-f251-433e-853f-59e8d13d4170 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:28.148210 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d39b4c-b9a7-40a6-9dc9-828f20a0d27a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:28.148394 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:28.150746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d39b4c-b9a7-40a6-9dc9-828f20a0d27a None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:28.242405 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd96ecd-f251-433e-853f-59e8d13d4170 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:28.255619 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:28.267119 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 320/655] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:56:28.267119 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d39b4c-b9a7-40a6-9dc9-828f20a0d27a None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:28.285418 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e95ea-f70a-4dff-b9ff-9827d6a9e011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:28.286028 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e95ea-f70a-4dff-b9ff-9827d6a9e011 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:28.286310 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e95ea-f70a-4dff-b9ff-9827d6a9e011 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:28.286584 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e95ea-f70a-4dff-b9ff-9827d6a9e011 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:28.287258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e95ea-f70a-4dff-b9ff-9827d6a9e011 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:28.287565 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e95ea-f70a-4dff-b9ff-9827d6a9e011 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:28.288979 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e95ea-f70a-4dff-b9ff-9827d6a9e011 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:28.300729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 336/656] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:28.305216 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39344183-2a16-40c4-9dc0-82e845d1180f None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:28.305216 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39344183-2a16-40c4-9dc0-82e845d1180f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:28.305216 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39344183-2a16-40c4-9dc0-82e845d1180f None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:28.305216 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 337/657] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:28.305216 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39344183-2a16-40c4-9dc0-82e845d1180f None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:28.378826 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d39b4c-b9a7-40a6-9dc9-828f20a0d27a None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:28.404417 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:28.414804 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e803d65-ec56-4d45-8b75-61f232486784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:28.415512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e803d65-ec56-4d45-8b75-61f232486784 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:28.415859 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e803d65-ec56-4d45-8b75-61f232486784 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:28.416200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e803d65-ec56-4d45-8b75-61f232486784 None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:28.416994 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e803d65-ec56-4d45-8b75-61f232486784 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:28.417368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e803d65-ec56-4d45-8b75-61f232486784 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:28.418787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e803d65-ec56-4d45-8b75-61f232486784 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:28.426197 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39344183-2a16-40c4-9dc0-82e845d1180f None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:28.436361 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 321/658] 10.222.0.36 () {56 vars in 1139 bytes} [Fri Sep 2 08:56:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:28.437179 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e803d65-ec56-4d45-8b75-61f232486784 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:28.467023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa0cf7-8261-4fa0-9ba7-e05280128bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:28.467785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa0cf7-8261-4fa0-9ba7-e05280128bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:28.468222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa0cf7-8261-4fa0-9ba7-e05280128bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:28.468499 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa0cf7-8261-4fa0-9ba7-e05280128bb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:28.478869 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa0cf7-8261-4fa0-9ba7-e05280128bb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:28.480346 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa0cf7-8261-4fa0-9ba7-e05280128bb6 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:28.490279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:28.495186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-04fa0cf7-8261-4fa0-9ba7-e05280128bb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:28.495186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:28.495186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:28.495186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:28.495186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:28.495186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:28.495186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:28.495186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:28.495186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:28.495186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:28.495186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:28.495186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:28.495186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:28.495186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:28.495186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:28.495186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:28.496165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:28.496165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:28.496165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:28.496165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:28.496165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:28.496165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:28.496165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:28.496165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:28.496165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:28.496165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:28.496165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:28.496165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:28.496165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:28.496165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:28.496165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:28.496165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:28.496165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:28.497068 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:28.497068 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:28.497068 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:28.497068 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 338/659] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:28.497573 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66df410-3a8d-4205-a64b-bef4bfe057bb None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:28.497666 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66df410-3a8d-4205-a64b-bef4bfe057bb None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:28.497892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66df410-3a8d-4205-a64b-bef4bfe057bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:28.515667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-d66df410-3a8d-4205-a64b-bef4bfe057bb None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:28.516832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-d66df410-3a8d-4205-a64b-bef4bfe057bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:28.547275 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e803d65-ec56-4d45-8b75-61f232486784 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:28.582932 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5841d-182c-4e02-846f-cac8a1e4d564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:28.583726 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5841d-182c-4e02-846f-cac8a1e4d564 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:28.591235 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 339/660] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:28.591361 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5841d-182c-4e02-846f-cac8a1e4d564 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:28.591361 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5841d-182c-4e02-846f-cac8a1e4d564 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:28.591520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66df410-3a8d-4205-a64b-bef4bfe057bb None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:28.600930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5841d-182c-4e02-846f-cac8a1e4d564 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:28.602422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5841d-182c-4e02-846f-cac8a1e4d564 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:28.618846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-5fc5841d-182c-4e02-846f-cac8a1e4d564 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 08:56:28.618846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:28.618846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:28.618846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:28.618846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:28.618846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:28.618846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:28.618846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:28.618846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:28.618846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:28.618846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:28.618846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:28.618846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:28.618846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:28.618846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:28.618846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:28.620048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:28.620048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:28.620048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:28.620048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:28.620048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:28.620048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:28.620048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:28.620048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:28.620048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:28.620048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:28.620048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:28.620048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:28.620048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:28.620048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:28.620048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:28.620048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:28.620048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:28.620048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:28.621060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:28.621060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:28.621060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:28.621060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:28.621060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:28.621060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:28.621060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:28.621060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:28.621060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 08:56:28.621060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:28.621060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 322/661] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:56:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:28.621647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5841d-182c-4e02-846f-cac8a1e4d564 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:28.661296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:28.704763 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66df410-3a8d-4205-a64b-bef4bfe057bb None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:28.744806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40afdb-86c0-48d4-9ce6-5da42d6ef449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:28.745497 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40afdb-86c0-48d4-9ce6-5da42d6ef449 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:28.745734 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40afdb-86c0-48d4-9ce6-5da42d6ef449 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:28.746006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40afdb-86c0-48d4-9ce6-5da42d6ef449 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:28.760614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5841d-182c-4e02-846f-cac8a1e4d564 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:28.775752 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:28.799229 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fb78c-1fba-4976-9970-c465f677cdc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:28.799864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fb78c-1fba-4976-9970-c465f677cdc1 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:28.800717 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fb78c-1fba-4976-9970-c465f677cdc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:28.800977 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fb78c-1fba-4976-9970-c465f677cdc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:28.801621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38fb78c-1fba-4976-9970-c465f677cdc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:28.801920 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38fb78c-1fba-4976-9970-c465f677cdc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:28.803327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38fb78c-1fba-4976-9970-c465f677cdc1 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:28.819907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 323/662] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:28.819907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fb78c-1fba-4976-9970-c465f677cdc1 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:28.847232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40afdb-86c0-48d4-9ce6-5da42d6ef449 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:28.850283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40afdb-86c0-48d4-9ce6-5da42d6ef449 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:28.918296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:28.925852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fb78c-1fba-4976-9970-c465f677cdc1 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:28.962336 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 340/663] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:56:28.963302 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40afdb-86c0-48d4-9ce6-5da42d6ef449 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:28.963815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758f2d7-758a-402b-94ea-d41caca075a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:28.964665 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758f2d7-758a-402b-94ea-d41caca075a9 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:28.965146 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758f2d7-758a-402b-94ea-d41caca075a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:28.965618 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758f2d7-758a-402b-94ea-d41caca075a9 None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:28.966810 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758f2d7-758a-402b-94ea-d41caca075a9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:28.967138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758f2d7-758a-402b-94ea-d41caca075a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:28.968578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7758f2d7-758a-402b-94ea-d41caca075a9 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:28.987201 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 324/664] 10.222.0.36 () {56 vars in 1137 bytes} [Fri Sep 2 08:56:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:28.988336 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758f2d7-758a-402b-94ea-d41caca075a9 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:29.025352 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:29.069341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40afdb-86c0-48d4-9ce6-5da42d6ef449 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:29.093831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758f2d7-758a-402b-94ea-d41caca075a9 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:29.103417 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832e0e9-acd8-4215-b59f-858611a31045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:29.104036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832e0e9-acd8-4215-b59f-858611a31045 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:29.104307 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832e0e9-acd8-4215-b59f-858611a31045 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:29.104574 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832e0e9-acd8-4215-b59f-858611a31045 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:29.110889 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e832e0e9-acd8-4215-b59f-858611a31045 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:29.112334 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e832e0e9-acd8-4215-b59f-858611a31045 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:29.123243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-e832e0e9-acd8-4215-b59f-858611a31045 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:29.123243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:29.123243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:29.123243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:29.123243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:29.123243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:29.123243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:29.123243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:29.123243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:29.123243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:29.123243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:29.123243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:29.123243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:29.123243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:29.123243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:29.124132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:29.124132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:29.124132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:29.124132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:29.124132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:29.124132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:29.124132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:29.124132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:29.124132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:29.124132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:29.124132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:29.124132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:29.124132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:29.124132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:29.124132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:29.124132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:29.124132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:29.125036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:29.125036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:29.125036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:29.125036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:29.125036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:29.125036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:29.125036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:29.125036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:29.125036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 341/665] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:29.125036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832e0e9-acd8-4215-b59f-858611a31045 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:29.132159 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6421b-1336-4af0-b8dd-9dedeeb1f502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:29.132788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6421b-1336-4af0-b8dd-9dedeeb1f502 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:29.133072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6421b-1336-4af0-b8dd-9dedeeb1f502 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:29.133343 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6421b-1336-4af0-b8dd-9dedeeb1f502 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:29.143454 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6421b-1336-4af0-b8dd-9dedeeb1f502 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:29.148433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:29.148770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-71b6421b-1336-4af0-b8dd-9dedeeb1f502 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:56:29.148770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:29.148770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:29.148770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:29.148770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:29.148770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:29.148770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:29.148770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:29.148770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:29.148770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:29.148770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:29.148770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:29.148770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:29.148770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:29.148770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:29.148770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:29.149657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:29.149657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:29.149657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:29.149657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:29.149657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:29.149657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:29.149657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:29.149657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:29.149657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:29.149657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:29.149657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:29.149657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:29.149657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:29.149657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:29.149657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:29.149657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:29.150506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:29.150506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:29.150506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:29.150506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:29.150506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:29.150506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:29.150506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:29.150506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:29.150506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:29.150506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:29.150506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:56:29.150506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:29.150506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 325/666] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:56:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:29.151236 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6421b-1336-4af0-b8dd-9dedeeb1f502 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:29.226762 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832e0e9-acd8-4215-b59f-858611a31045 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:29.240646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:29.250006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6421b-1336-4af0-b8dd-9dedeeb1f502 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:29.259265 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d2336-80bd-4c3d-aa66-7dee423a4f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:29.259887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d2336-80bd-4c3d-aa66-7dee423a4f0f None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:29.260158 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d2336-80bd-4c3d-aa66-7dee423a4f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:29.260425 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d2336-80bd-4c3d-aa66-7dee423a4f0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:29.270624 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d2336-80bd-4c3d-aa66-7dee423a4f0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:29.272070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d2336-80bd-4c3d-aa66-7dee423a4f0f None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:29.283627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8affd23c-0cc2-41aa-b19f-51d8ae1364d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:29.284184 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8affd23c-0cc2-41aa-b19f-51d8ae1364d1 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:29.284486 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8affd23c-0cc2-41aa-b19f-51d8ae1364d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:29.284753 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8affd23c-0cc2-41aa-b19f-51d8ae1364d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:29.286410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-e22d2336-80bd-4c3d-aa66-7dee423a4f0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:29.286410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:29.286410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:29.286410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:29.286410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:29.286410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:29.286410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:29.286410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:29.286410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:29.286410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:29.286410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:29.286410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:29.286410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:29.286410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:29.286410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:29.286410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:29.287544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:29.287544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:29.287544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:29.287544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:29.287544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:29.287544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:29.287544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:29.287544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:29.287544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:29.287544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:29.287544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:29.287544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:29.287544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:29.287544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:29.287544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:29.287544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:29.287544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:29.288663 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:29.288663 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:29.288663 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:29.292086 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 342/667] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:29.293171 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99e264-31af-45d7-9a06-ea86546d4e95 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:29.293455 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99e264-31af-45d7-9a06-ea86546d4e95 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:29.293730 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99e264-31af-45d7-9a06-ea86546d4e95 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:29.294960 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 343/668] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:29.295670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99e264-31af-45d7-9a06-ea86546d4e95 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:29.356548 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8affd23c-0cc2-41aa-b19f-51d8ae1364d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:29.358014 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8affd23c-0cc2-41aa-b19f-51d8ae1364d1 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:29.373413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:29.399913 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99e264-31af-45d7-9a06-ea86546d4e95 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:29.432438 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ceb22-5ae6-440c-87cc-f760f19fb30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:29.433034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ceb22-5ae6-440c-87cc-f760f19fb30a None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:29.433308 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ceb22-5ae6-440c-87cc-f760f19fb30a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:29.433578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ceb22-5ae6-440c-87cc-f760f19fb30a None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:29.434506 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ceb22-5ae6-440c-87cc-f760f19fb30a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:29.434801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ceb22-5ae6-440c-87cc-f760f19fb30a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:29.436249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2ceb22-5ae6-440c-87cc-f760f19fb30a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:29.445497 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:29.448586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 344/669] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:29.449365 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ceb22-5ae6-440c-87cc-f760f19fb30a None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:29.465917 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 326/670] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:56:29.465917 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8affd23c-0cc2-41aa-b19f-51d8ae1364d1 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:29.549389 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ceb22-5ae6-440c-87cc-f760f19fb30a None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:29.557494 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8affd23c-0cc2-41aa-b19f-51d8ae1364d1 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:29.582748 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7220e80-0f70-410f-864d-5727e6169434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:29.583450 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7220e80-0f70-410f-864d-5727e6169434 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:29.583722 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7220e80-0f70-410f-864d-5727e6169434 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:29.583987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7220e80-0f70-410f-864d-5727e6169434 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:29.584626 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7220e80-0f70-410f-864d-5727e6169434 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:29.584919 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7220e80-0f70-410f-864d-5727e6169434 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:29.586277 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7220e80-0f70-410f-864d-5727e6169434 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:29.587604 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:29.591468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c3d58-9d3a-4aea-9c22-9bc9739d3ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:29.592076 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c3d58-9d3a-4aea-9c22-9bc9739d3ad7 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:29.592354 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c3d58-9d3a-4aea-9c22-9bc9739d3ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:29.592631 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734c3d58-9d3a-4aea-9c22-9bc9739d3ad7 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:29.593269 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734c3d58-9d3a-4aea-9c22-9bc9739d3ad7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:29.593572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734c3d58-9d3a-4aea-9c22-9bc9739d3ad7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:29.594978 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734c3d58-9d3a-4aea-9c22-9bc9739d3ad7 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:29.595209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 345/671] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:56:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:29.596433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4eb9d8-e2f2-4c24-8aa8-da50d1adb127 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:29.596721 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4eb9d8-e2f2-4c24-8aa8-da50d1adb127 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:29.597015 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4eb9d8-e2f2-4c24-8aa8-da50d1adb127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:29.606260 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 327/672] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:29.607047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c3d58-9d3a-4aea-9c22-9bc9739d3ad7 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:29.616021 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-7c4eb9d8-e2f2-4c24-8aa8-da50d1adb127 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:29.617195 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-7c4eb9d8-e2f2-4c24-8aa8-da50d1adb127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:29.656076 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:29.681652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 346/673] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:29.682637 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4eb9d8-e2f2-4c24-8aa8-da50d1adb127 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:29.707242 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734c3d58-9d3a-4aea-9c22-9bc9739d3ad7 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:29.744073 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111508e3-3f9f-4d9d-a826-d3f0a3eeca30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:29.744671 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111508e3-3f9f-4d9d-a826-d3f0a3eeca30 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:29.745057 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111508e3-3f9f-4d9d-a826-d3f0a3eeca30 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:29.745327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111508e3-3f9f-4d9d-a826-d3f0a3eeca30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:29.746160 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111508e3-3f9f-4d9d-a826-d3f0a3eeca30 None admin] RBAC: Adding query filter params (role.id=637ed3cd3d484839ab21de02c2bbe293, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=2474d89f10764afbbc5e740dccaae4df) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:29.746469 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111508e3-3f9f-4d9d-a826-d3f0a3eeca30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:29.748019 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111508e3-3f9f-4d9d-a826-d3f0a3eeca30 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:29.755326 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 328/674] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:56:29 2022] GET /identity/v3/role_assignments?user.id=2474d89f10764afbbc5e740dccaae4df&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=637ed3cd3d484839ab21de02c2bbe293 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:29.756118 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111508e3-3f9f-4d9d-a826-d3f0a3eeca30 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:29.804797 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4eb9d8-e2f2-4c24-8aa8-da50d1adb127 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:29.843762 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:29.855384 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef670fac-7ad2-4816-badf-b4381d7d68b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:29.856716 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef670fac-7ad2-4816-badf-b4381d7d68b2 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:29.857088 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef670fac-7ad2-4816-badf-b4381d7d68b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:29.857711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef670fac-7ad2-4816-badf-b4381d7d68b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:29.873177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef670fac-7ad2-4816-badf-b4381d7d68b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:29.875307 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef670fac-7ad2-4816-badf-b4381d7d68b2 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:29.880632 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111508e3-3f9f-4d9d-a826-d3f0a3eeca30 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:29.897143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-ef670fac-7ad2-4816-badf-b4381d7d68b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:29.897143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:29.897143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:29.897143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:29.897143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:29.897143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:29.897143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:29.897143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:29.897143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:29.897143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:29.897143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:29.897143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:29.897143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:29.897143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:29.897143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:29.897143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:29.898249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:29.898249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:29.898249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:29.898249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:29.898249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:29.898249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:29.898249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:29.898249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:29.898249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:29.898249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:29.898249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:29.898249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:29.898249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:29.898249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:29.898249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:29.898249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:29.898249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:29.899523 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:29.899523 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:29.899523 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:29.899887 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:29.899966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 347/675] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:29.900858 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef670fac-7ad2-4816-badf-b4381d7d68b2 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:29.931014 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042add7-7edd-43df-a146-153c2a3e0dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:29.931014 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042add7-7edd-43df-a146-153c2a3e0dfc None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:29.931686 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042add7-7edd-43df-a146-153c2a3e0dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:29.931686 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042add7-7edd-43df-a146-153c2a3e0dfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:29.948162 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2042add7-7edd-43df-a146-153c2a3e0dfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:29.949510 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2042add7-7edd-43df-a146-153c2a3e0dfc None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:29.964921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-2042add7-7edd-43df-a146-153c2a3e0dfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:29.964921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:29.964921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:29.964921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:29.964921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:29.964921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:29.964921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:29.964921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:29.964921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:29.964921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:29.964921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:29.964921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:29.964921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:29.964921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:29.964921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:29.964921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:29.966100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:29.966100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:29.966100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:29.966100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:29.966100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:29.966100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:29.966100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:29.966100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:29.966100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:29.966100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:29.966100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:29.966100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:29.966100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:29.966100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:29.966100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:29.966100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:29.966100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:29.967223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:29.967223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:29.967223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:29.967223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 329/676] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:29.967223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042add7-7edd-43df-a146-153c2a3e0dfc None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:30.040362 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef670fac-7ad2-4816-badf-b4381d7d68b2 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:30.079114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069426c1-e7b0-43b2-93e5-059689c76673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:30.079736 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2042add7-7edd-43df-a146-153c2a3e0dfc None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:30.079836 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069426c1-e7b0-43b2-93e5-059689c76673 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:30.080150 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069426c1-e7b0-43b2-93e5-059689c76673 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:30.080488 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069426c1-e7b0-43b2-93e5-059689c76673 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:30.088420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069426c1-e7b0-43b2-93e5-059689c76673 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:30.089934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069426c1-e7b0-43b2-93e5-059689c76673 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:30.101911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-069426c1-e7b0-43b2-93e5-059689c76673 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:30.101911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:30.101911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:30.101911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:30.101911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:30.101911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:30.101911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:30.101911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:30.101911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:30.101911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:30.101911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:30.101911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:30.101911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:30.101911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:30.101911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:30.103001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:30.103001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:30.103001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:30.103001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:30.103001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:30.103001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:30.103001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:30.103001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:30.103001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:30.103001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:30.103001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:30.103001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:30.103001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:30.103001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:30.103001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:30.103001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:30.103001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:30.104123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:30.104123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:30.104123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:30.104123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:30.104123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:30.104123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:30.104123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:30.104123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:30.108468 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 348/677] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:30.109241 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069426c1-e7b0-43b2-93e5-059689c76673 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:30.118430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78368f61-8e87-4900-a79e-de90a371cce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:30.119192 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78368f61-8e87-4900-a79e-de90a371cce9 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:30.119548 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78368f61-8e87-4900-a79e-de90a371cce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:30.119868 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78368f61-8e87-4900-a79e-de90a371cce9 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:30.120519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78368f61-8e87-4900-a79e-de90a371cce9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:30.120821 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78368f61-8e87-4900-a79e-de90a371cce9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:30.122204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78368f61-8e87-4900-a79e-de90a371cce9 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:30.134116 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 330/678] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:30.134942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78368f61-8e87-4900-a79e-de90a371cce9 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:30.158692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:30.166662 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:30.234715 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069426c1-e7b0-43b2-93e5-059689c76673 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:30.243612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78368f61-8e87-4900-a79e-de90a371cce9 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:30.271056 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f3f02-09db-4233-b9b2-77aac941fbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:30.271710 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f3f02-09db-4233-b9b2-77aac941fbd4 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:30.271984 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f3f02-09db-4233-b9b2-77aac941fbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:30.272253 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f3f02-09db-4233-b9b2-77aac941fbd4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:30.279161 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0938bfe-656d-4ef8-94c9-2fb9f01ae7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:30.279772 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0938bfe-656d-4ef8-94c9-2fb9f01ae7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:30.280057 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0938bfe-656d-4ef8-94c9-2fb9f01ae7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:30.280337 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0938bfe-656d-4ef8-94c9-2fb9f01ae7e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:30.280976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0938bfe-656d-4ef8-94c9-2fb9f01ae7e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:30.281280 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0938bfe-656d-4ef8-94c9-2fb9f01ae7e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:30.282705 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0938bfe-656d-4ef8-94c9-2fb9f01ae7e7 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:30.283179 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f3f02-09db-4233-b9b2-77aac941fbd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:30.289599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-bd0f3f02-09db-4233-b9b2-77aac941fbd4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:30.289599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:30.289599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:30.289599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:30.289599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:30.289599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:30.289599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:30.289599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:30.289599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:30.289599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:30.289599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:30.289599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:30.289599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:30.289599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:30.289599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:30.289599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:30.290586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:30.290586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:30.290586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:30.290586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:30.290586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:30.290586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:30.290586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:30.290586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:30.290586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:30.290586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:30.290586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:30.290586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:30.290586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:30.290586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:30.290586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:30.290586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:30.296932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:30.296932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:30.296932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:30.296932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:30.296932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:30.296932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:30.296932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:30.296932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:30.296932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:30.296932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:30.296932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:30.296932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:30.296932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 349/679] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:56:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:30.296932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f3f02-09db-4233-b9b2-77aac941fbd4 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:30.297682 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 331/680] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:30.297682 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0938bfe-656d-4ef8-94c9-2fb9f01ae7e7 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:30.393190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:30.408255 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:30.409966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0938bfe-656d-4ef8-94c9-2fb9f01ae7e7 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:30.413245 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f3f02-09db-4233-b9b2-77aac941fbd4 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:30.444691 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626ed23-3e74-4838-8c69-0883c831474b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:30.445325 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626ed23-3e74-4838-8c69-0883c831474b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:30.445597 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626ed23-3e74-4838-8c69-0883c831474b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:30.445867 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626ed23-3e74-4838-8c69-0883c831474b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:30.449883 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5120a-bb8f-4cf9-8f58-0d48721d5a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:30.450655 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5120a-bb8f-4cf9-8f58-0d48721d5a1c None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:30.451135 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5120a-bb8f-4cf9-8f58-0d48721d5a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:30.451575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5120a-bb8f-4cf9-8f58-0d48721d5a1c None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:30.452416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c5120a-bb8f-4cf9-8f58-0d48721d5a1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:30.452644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626ed23-3e74-4838-8c69-0883c831474b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:30.453021 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c5120a-bb8f-4cf9-8f58-0d48721d5a1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:30.454076 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8626ed23-3e74-4838-8c69-0883c831474b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:30.454774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c5120a-bb8f-4cf9-8f58-0d48721d5a1c None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:30.466398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-8626ed23-3e74-4838-8c69-0883c831474b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:30.466398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:30.466398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:30.466398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:30.466398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:30.466398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:30.466398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:30.466398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:30.466398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:30.466398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:30.466398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:30.466398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:30.466398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:30.466398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:30.466398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:30.467512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:30.467512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:30.467512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:30.467512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:30.467512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:30.467512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:30.467512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:30.467512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:30.467512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:30.467512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:30.467512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:30.467512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:30.467512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:30.467512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:30.467512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:30.467512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:30.467512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:30.468420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:30.468420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:30.468420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:30.468420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:30.468420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:30.468420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:30.468420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:30.468420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:30.468420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 332/681] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:30.468420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626ed23-3e74-4838-8c69-0883c831474b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:30.470775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 350/682] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:30.471839 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5120a-bb8f-4cf9-8f58-0d48721d5a1c None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:30.574351 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626ed23-3e74-4838-8c69-0883c831474b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:30.576549 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5120a-bb8f-4cf9-8f58-0d48721d5a1c None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:30.614089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2cc82f-6a0e-4f22-9d74-c80939e082a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:30.614951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cc82f-6a0e-4f22-9d74-c80939e082a8 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:30.615236 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cc82f-6a0e-4f22-9d74-c80939e082a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:30.615602 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cc82f-6a0e-4f22-9d74-c80939e082a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:30.616492 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2cc82f-6a0e-4f22-9d74-c80939e082a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:30.616796 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2cc82f-6a0e-4f22-9d74-c80939e082a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:30.618165 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2cc82f-6a0e-4f22-9d74-c80939e082a8 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:30.618408 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8dfdd-edae-46dd-9ba7-e5dc56af8f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:30.619040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8dfdd-edae-46dd-9ba7-e5dc56af8f7f None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:30.619350 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8dfdd-edae-46dd-9ba7-e5dc56af8f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:30.619791 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a8dfdd-edae-46dd-9ba7-e5dc56af8f7f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:30.627962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a8dfdd-edae-46dd-9ba7-e5dc56af8f7f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:30.629384 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a8dfdd-edae-46dd-9ba7-e5dc56af8f7f None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:30.631270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 333/683] 10.222.0.36 () {56 vars in 1103 bytes} [Fri Sep 2 08:56:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:30.632111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2cc82f-6a0e-4f22-9d74-c80939e082a8 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:30.641674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-b4a8dfdd-edae-46dd-9ba7-e5dc56af8f7f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 08:56:30.641674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:30.641674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:30.641674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:30.641674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:30.641674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:30.641674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:30.641674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:30.641674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:30.641674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:30.641674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:30.641674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:30.641674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:30.641674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:30.641674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:30.641674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:30.642692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:30.642692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:30.642692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:30.642692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:30.642692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:30.642692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:30.642692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:30.642692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:30.642692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:30.642692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:30.642692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:30.642692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:30.642692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:30.642692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:30.642692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:30.642692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:30.642692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:30.642692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:30.644306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:30.644306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:30.644306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:30.644306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:30.644306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:30.644306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:30.644306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:30.644306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:30.644306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 08:56:30.644306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:30.644306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 351/684] 10.222.0.36 () {56 vars in 1093 bytes} [Fri Sep 2 08:56:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:30.644306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8dfdd-edae-46dd-9ba7-e5dc56af8f7f None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:30.646252 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:30.665006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:30.746029 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2cc82f-6a0e-4f22-9d74-c80939e082a8 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:30.754175 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a8dfdd-edae-46dd-9ba7-e5dc56af8f7f None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:30.781269 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085a22a-99da-4e9c-858e-38b030c5f8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:30.781978 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085a22a-99da-4e9c-858e-38b030c5f8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:30.782252 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085a22a-99da-4e9c-858e-38b030c5f8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:30.782623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085a22a-99da-4e9c-858e-38b030c5f8a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:30.783322 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085a22a-99da-4e9c-858e-38b030c5f8a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:30.783625 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085a22a-99da-4e9c-858e-38b030c5f8a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:30.785067 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085a22a-99da-4e9c-858e-38b030c5f8a2 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:30.787677 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b90d8-87f8-4e3f-b31f-f1305de2d16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:30.788279 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b90d8-87f8-4e3f-b31f-f1305de2d16c None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:30.788558 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b90d8-87f8-4e3f-b31f-f1305de2d16c None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:30.788830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b90d8-87f8-4e3f-b31f-f1305de2d16c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:30.789551 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b90d8-87f8-4e3f-b31f-f1305de2d16c None admin] RBAC: Adding query filter params (role.id=637ed3cd3d484839ab21de02c2bbe293, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=07a825fddbc04f918bcd631976038cf9) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:30.789858 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b90d8-87f8-4e3f-b31f-f1305de2d16c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:30.791302 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b90d8-87f8-4e3f-b31f-f1305de2d16c None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:30.797155 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 352/685] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:56:30 2022] GET /identity/v3/role_assignments?user.id=07a825fddbc04f918bcd631976038cf9&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=637ed3cd3d484839ab21de02c2bbe293 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:30.797946 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b90d8-87f8-4e3f-b31f-f1305de2d16c None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:30.799119 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 334/686] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:30.800151 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085a22a-99da-4e9c-858e-38b030c5f8a2 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:30.882525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:30.903102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:30.925071 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b90d8-87f8-4e3f-b31f-f1305de2d16c None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:30.935443 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085a22a-99da-4e9c-858e-38b030c5f8a2 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:30.971126 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc905f-8688-4c89-bdf4-82a2c7ae0af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:30.971809 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc905f-8688-4c89-bdf4-82a2c7ae0af3 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:30.972098 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc905f-8688-4c89-bdf4-82a2c7ae0af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:30.972381 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc905f-8688-4c89-bdf4-82a2c7ae0af3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:30.983355 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc905f-8688-4c89-bdf4-82a2c7ae0af3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:30.983590 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94083eef-5b51-4e4d-b9bf-ac48e2e891c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:30.984163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94083eef-5b51-4e4d-b9bf-ac48e2e891c3 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:30.984446 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94083eef-5b51-4e4d-b9bf-ac48e2e891c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:30.984723 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94083eef-5b51-4e4d-b9bf-ac48e2e891c3 None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:30.984829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc905f-8688-4c89-bdf4-82a2c7ae0af3 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:30.985462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94083eef-5b51-4e4d-b9bf-ac48e2e891c3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:30.985770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94083eef-5b51-4e4d-b9bf-ac48e2e891c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:30.987477 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94083eef-5b51-4e4d-b9bf-ac48e2e891c3 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:31.000059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-addc905f-8688-4c89-bdf4-82a2c7ae0af3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:31.000059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:31.000059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:31.000059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:31.000059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:31.000059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:31.000059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:31.000059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:31.000059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:31.000059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:31.000059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:31.000059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:31.000059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:31.000059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:31.000059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:31.000059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:31.001048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:31.001048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:31.001048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:31.001048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:31.001048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:31.001048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:31.001048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:31.001048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:31.001048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:31.001048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:31.001048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:31.001048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:31.001048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:31.001048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:31.001048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:31.001048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:31.001048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:31.003656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:31.003656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:31.003656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:31.003656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 353/687] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:31.003656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc905f-8688-4c89-bdf4-82a2c7ae0af3 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:31.009377 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 335/688] 10.222.0.36 () {56 vars in 1133 bytes} [Fri Sep 2 08:56:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:31.010208 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94083eef-5b51-4e4d-b9bf-ac48e2e891c3 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:31.138315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94083eef-5b51-4e4d-b9bf-ac48e2e891c3 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:31.144930 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc905f-8688-4c89-bdf4-82a2c7ae0af3 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:31.173041 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:31.184516 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f9bd57-e66d-4390-9448-4679b939c4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:31.185129 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f9bd57-e66d-4390-9448-4679b939c4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:31.185413 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f9bd57-e66d-4390-9448-4679b939c4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:31.185687 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f9bd57-e66d-4390-9448-4679b939c4e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:31.186336 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f9bd57-e66d-4390-9448-4679b939c4e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:31.186638 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f9bd57-e66d-4390-9448-4679b939c4e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:31.188109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f9bd57-e66d-4390-9448-4679b939c4e7 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:31.189353 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e53df-ef72-4881-9c94-11734e8d4899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:31.190184 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e53df-ef72-4881-9c94-11734e8d4899 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:31.191258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e53df-ef72-4881-9c94-11734e8d4899 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:31.191726 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e53df-ef72-4881-9c94-11734e8d4899 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:31.192636 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e53df-ef72-4881-9c94-11734e8d4899 None admin] RBAC: Adding query filter params (role.id=637ed3cd3d484839ab21de02c2bbe293, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=f8370135a78f414f9184c883ccb92767) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:31.193153 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e53df-ef72-4881-9c94-11734e8d4899 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:31.194801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e53df-ef72-4881-9c94-11734e8d4899 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:31.196887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:31.203322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 354/689] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:56:31 2022] GET /identity/v3/role_assignments?user.id=f8370135a78f414f9184c883ccb92767&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=637ed3cd3d484839ab21de02c2bbe293 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:31.204470 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e53df-ef72-4881-9c94-11734e8d4899 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:31.207902 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 336/690] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:31.213420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f9bd57-e66d-4390-9448-4679b939c4e7 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:31.398973 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e53df-ef72-4881-9c94-11734e8d4899 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:31.418910 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f9bd57-e66d-4390-9448-4679b939c4e7 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:31.443280 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61636e-177d-4136-babb-3ff9b9c3cd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:31.443889 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61636e-177d-4136-babb-3ff9b9c3cd37 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:31.444166 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61636e-177d-4136-babb-3ff9b9c3cd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:31.444501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61636e-177d-4136-babb-3ff9b9c3cd37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:31.465709 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d843f-0f1d-4d08-bd4e-d3dd7542a9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:31.466930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d843f-0f1d-4d08-bd4e-d3dd7542a9c9 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:31.467466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d843f-0f1d-4d08-bd4e-d3dd7542a9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:31.467956 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388d843f-0f1d-4d08-bd4e-d3dd7542a9c9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:31.478670 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388d843f-0f1d-4d08-bd4e-d3dd7542a9c9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:31.480999 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388d843f-0f1d-4d08-bd4e-d3dd7542a9c9 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:31.496878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-388d843f-0f1d-4d08-bd4e-d3dd7542a9c9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 02 08:56:31.496878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:31.496878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:31.496878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:31.496878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:31.496878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:31.496878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:31.496878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:31.496878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:31.496878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:31.496878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:31.496878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:31.496878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:31.496878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:31.496878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:31.496878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:31.497856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:31.497856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:31.497856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:31.497856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:31.497856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:31.497856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:31.497856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:31.497856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:31.497856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:31.497856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:31.497856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:31.497856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:31.497856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:31.497856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:31.497856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:31.497856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:31.497856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:31.497856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:31.498790 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:31.498790 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:31.498790 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:31.498790 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:31.498790 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:31.498790 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:31.498790 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:31.498790 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:31.498790 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 02 08:56:31.498790 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:31.505164 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 337/691] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:56:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:31.506817 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d843f-0f1d-4d08-bd4e-d3dd7542a9c9 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:31.558020 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:31.560050 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:31.565084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61636e-177d-4136-babb-3ff9b9c3cd37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:31.567394 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61636e-177d-4136-babb-3ff9b9c3cd37 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:31.627753 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388d843f-0f1d-4d08-bd4e-d3dd7542a9c9 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:31.669347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2aee1e-585a-4db8-9d02-7835a03551a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:31.669983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2aee1e-585a-4db8-9d02-7835a03551a2 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:31.670333 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2aee1e-585a-4db8-9d02-7835a03551a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:31.670672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2aee1e-585a-4db8-9d02-7835a03551a2 None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:31.671608 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2aee1e-585a-4db8-9d02-7835a03551a2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:31.671984 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2aee1e-585a-4db8-9d02-7835a03551a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:31.673576 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2aee1e-585a-4db8-9d02-7835a03551a2 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:31.694820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 355/692] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 08:56:31.699908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61636e-177d-4136-babb-3ff9b9c3cd37 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:31.702556 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 338/693] 10.222.0.36 () {56 vars in 1135 bytes} [Fri Sep 2 08:56:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:31.708331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2aee1e-585a-4db8-9d02-7835a03551a2 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:31.807925 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:31.812603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61636e-177d-4136-babb-3ff9b9c3cd37 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:31.856236 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64b5af-cb35-45ee-8a69-52122ae1ab91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:31.857517 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64b5af-cb35-45ee-8a69-52122ae1ab91 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:31.857996 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64b5af-cb35-45ee-8a69-52122ae1ab91 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:31.858474 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64b5af-cb35-45ee-8a69-52122ae1ab91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:31.871648 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2aee1e-585a-4db8-9d02-7835a03551a2 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:31.872581 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed64b5af-cb35-45ee-8a69-52122ae1ab91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:31.875237 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed64b5af-cb35-45ee-8a69-52122ae1ab91 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:31.879401 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:31.894634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-ed64b5af-cb35-45ee-8a69-52122ae1ab91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:31.894634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:31.894634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:31.894634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:31.894634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:31.894634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:31.894634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:31.894634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:31.894634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:31.894634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:31.894634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:31.894634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:31.894634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:31.894634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:31.894634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:31.894634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:31.895670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:31.895670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:31.895670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:31.895670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:31.895670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:31.895670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:31.895670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:31.895670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:31.895670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:31.895670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:31.895670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:31.895670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:31.895670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:31.895670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:31.895670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:31.895670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:31.895670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:31.896566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:31.896566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:31.896566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:31.898728 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 356/694] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:31.900291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034d3c1-6628-40f7-a021-0dfef9b0c0d5 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:31.900770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034d3c1-6628-40f7-a021-0dfef9b0c0d5 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:31.901247 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034d3c1-6628-40f7-a021-0dfef9b0c0d5 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:31.903020 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 357/695] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:31.903931 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034d3c1-6628-40f7-a021-0dfef9b0c0d5 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:31.917172 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc45293-9fe5-4bf0-ab61-87ad214b6fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:31.917784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc45293-9fe5-4bf0-ab61-87ad214b6fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:31.918058 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc45293-9fe5-4bf0-ab61-87ad214b6fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:31.918325 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc45293-9fe5-4bf0-ab61-87ad214b6fe7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:31.991920 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc45293-9fe5-4bf0-ab61-87ad214b6fe7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:31.993395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc45293-9fe5-4bf0-ab61-87ad214b6fe7 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:32.013211 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034d3c1-6628-40f7-a021-0dfef9b0c0d5 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:32.055525 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272f72d-e770-488b-817a-58e53e50a7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:32.056919 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272f72d-e770-488b-817a-58e53e50a7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:32.057408 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272f72d-e770-488b-817a-58e53e50a7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:32.058079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272f72d-e770-488b-817a-58e53e50a7f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:32.058873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272f72d-e770-488b-817a-58e53e50a7f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:32.059715 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272f72d-e770-488b-817a-58e53e50a7f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:32.061889 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272f72d-e770-488b-817a-58e53e50a7f6 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:32.077513 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 358/696] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:32.078994 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1648a-0b9c-4c50-b096-3a136ee7a35e None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:32.079833 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1648a-0b9c-4c50-b096-3a136ee7a35e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:32.080258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f1648a-0b9c-4c50-b096-3a136ee7a35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:32.105141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-19f1648a-0b9c-4c50-b096-3a136ee7a35e None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:32.106874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-19f1648a-0b9c-4c50-b096-3a136ee7a35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:32.108858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 339/697] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 02 08:56:32.115120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc45293-9fe5-4bf0-ab61-87ad214b6fe7 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:32.116121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:32.117841 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:32.203109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 359/698] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:32.203109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1648a-0b9c-4c50-b096-3a136ee7a35e None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:32.255376 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc45293-9fe5-4bf0-ab61-87ad214b6fe7 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:32.317992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f1648a-0b9c-4c50-b096-3a136ee7a35e None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:32.354945 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7e48e-f3fb-4236-bbab-02f54ea100c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:32.355624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7e48e-f3fb-4236-bbab-02f54ea100c9 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:32.355900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7e48e-f3fb-4236-bbab-02f54ea100c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:32.356171 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7e48e-f3fb-4236-bbab-02f54ea100c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:32.363463 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c41b0-9c4a-4fda-8dc2-70e4e3b028f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:32.364489 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c41b0-9c4a-4fda-8dc2-70e4e3b028f3 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:32.364769 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c41b0-9c4a-4fda-8dc2-70e4e3b028f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:32.365042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c41b0-9c4a-4fda-8dc2-70e4e3b028f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:32.375172 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:32.376045 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb7e48e-f3fb-4236-bbab-02f54ea100c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:32.377411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb7e48e-f3fb-4236-bbab-02f54ea100c9 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:32.379121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c41b0-9c4a-4fda-8dc2-70e4e3b028f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:32.382106 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8c41b0-9c4a-4fda-8dc2-70e4e3b028f3 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:32.397366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-ddb7e48e-f3fb-4236-bbab-02f54ea100c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:32.397366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:32.397366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:32.397366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:32.397366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:32.397366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:32.397366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:32.397366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:32.397366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:32.397366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:32.397366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:32.397366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:32.397366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:32.397366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:32.397366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:32.398310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:32.398310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:32.398310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:32.398310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:32.398310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:32.398310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:32.398310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:32.398310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:32.398310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:32.398310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:32.398310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:32.398310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:32.398310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:32.398310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:32.398310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:32.398310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:32.398310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:32.399435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:32.399435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:32.399435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:32.399435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:32.399435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:32.399435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:32.399435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:32.399435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:32.402275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-ea8c41b0-9c4a-4fda-8dc2-70e4e3b028f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:32.402275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:32.402275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:32.402275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:32.402275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:32.402275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:32.402275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:32.402275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:32.402275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:32.402275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:32.402275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:32.402275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:32.402275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:32.402275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:32.402275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:32.402275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:32.403416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:32.403416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:32.403416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:32.403416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:32.403416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:32.403416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:32.403416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:32.403416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:32.403416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:32.403416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:32.403416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:32.403416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:32.403416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:32.403416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:32.403416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:32.403416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:32.403416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:32.404590 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:32.404590 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:32.404590 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:32.406227 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 360/699] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:32.410800 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 340/700] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:32.411868 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c41b0-9c4a-4fda-8dc2-70e4e3b028f3 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:32.412906 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7e48e-f3fb-4236-bbab-02f54ea100c9 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:32.511832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:32.555641 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7e48e-f3fb-4236-bbab-02f54ea100c9 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:32.600072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8c41b0-9c4a-4fda-8dc2-70e4e3b028f3 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:32.610923 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c239612-bf5c-4547-bf46-17d4f6cb6dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:32.611563 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c239612-bf5c-4547-bf46-17d4f6cb6dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:32.611886 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c239612-bf5c-4547-bf46-17d4f6cb6dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:32.612157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c239612-bf5c-4547-bf46-17d4f6cb6dc1 None admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:32.613365 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c239612-bf5c-4547-bf46-17d4f6cb6dc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:32.613665 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c239612-bf5c-4547-bf46-17d4f6cb6dc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:32.615177 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c239612-bf5c-4547-bf46-17d4f6cb6dc1 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:32.626909 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 341/701] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:32.627822 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c239612-bf5c-4547-bf46-17d4f6cb6dc1 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:32.639262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26386673-4661-4fca-91c0-6661707692d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:32.639884 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26386673-4661-4fca-91c0-6661707692d7 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:32.640166 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26386673-4661-4fca-91c0-6661707692d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:32.640495 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26386673-4661-4fca-91c0-6661707692d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:32.641148 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26386673-4661-4fca-91c0-6661707692d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:32.641450 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26386673-4661-4fca-91c0-6661707692d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:32.643083 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26386673-4661-4fca-91c0-6661707692d7 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:32.656702 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 361/702] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:32.657548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26386673-4661-4fca-91c0-6661707692d7 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:32.715778 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:32.756836 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c239612-bf5c-4547-bf46-17d4f6cb6dc1 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:32.770870 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26386673-4661-4fca-91c0-6661707692d7 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:32.789826 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:32.806340 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50510f-86d9-46c6-b371-70900be2f74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:32.807423 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50510f-86d9-46c6-b371-70900be2f74f None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:32.807936 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50510f-86d9-46c6-b371-70900be2f74f None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:32.808270 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a649c-37f1-494d-af13-c3cc685bf098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:32.808651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50510f-86d9-46c6-b371-70900be2f74f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:32.808892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a649c-37f1-494d-af13-c3cc685bf098 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:32.809428 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a649c-37f1-494d-af13-c3cc685bf098 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:32.809703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a649c-37f1-494d-af13-c3cc685bf098 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:32.810450 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a649c-37f1-494d-af13-c3cc685bf098 None admin] RBAC: Adding query filter params (role.id=a0a76de72fba4036bc1e3e64df717c7f, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=cce502a83f5343558c2136c0193e7d85) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:32.810764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a649c-37f1-494d-af13-c3cc685bf098 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:32.812464 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a649c-37f1-494d-af13-c3cc685bf098 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:32.824701 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50510f-86d9-46c6-b371-70900be2f74f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:32.824918 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 362/703] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:56:32 2022] GET /identity/v3/role_assignments?user.id=cce502a83f5343558c2136c0193e7d85&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=a0a76de72fba4036bc1e3e64df717c7f => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:32.825866 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a649c-37f1-494d-af13-c3cc685bf098 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:32.826830 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50510f-86d9-46c6-b371-70900be2f74f None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:32.845090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-4a50510f-86d9-46c6-b371-70900be2f74f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:32.845090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:32.845090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:32.845090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:32.845090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:32.845090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:32.845090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:32.845090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:32.845090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:32.845090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:32.845090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:32.845090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:32.845090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:32.845090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:32.845090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:32.845090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:32.846120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:32.846120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:32.846120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:32.846120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:32.846120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:32.846120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:32.846120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:32.846120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:32.846120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:32.846120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:32.846120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:32.846120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:32.846120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:32.846120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:32.846120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:32.846120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:32.846120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:32.847053 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:32.847053 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:32.847053 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:32.849301 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 342/704] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:32.856751 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50510f-86d9-46c6-b371-70900be2f74f None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:32.982052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a649c-37f1-494d-af13-c3cc685bf098 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:32.990858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50510f-86d9-46c6-b371-70900be2f74f None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:33.001408 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:33.024303 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea523c5b-b655-4af7-a77d-c35d75bd2f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:33.024861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea523c5b-b655-4af7-a77d-c35d75bd2f54 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:33.025136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea523c5b-b655-4af7-a77d-c35d75bd2f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:33.025405 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea523c5b-b655-4af7-a77d-c35d75bd2f54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:33.032535 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea523c5b-b655-4af7-a77d-c35d75bd2f54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:33.033975 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea523c5b-b655-4af7-a77d-c35d75bd2f54 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:33.044738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d14e5e-67ba-4af6-a5ea-d658b1683a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:33.045933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d14e5e-67ba-4af6-a5ea-d658b1683a6f None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:33.046181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-ea523c5b-b655-4af7-a77d-c35d75bd2f54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:33.046181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:33.046181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:33.046181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:33.046181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:33.046181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:33.046181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:33.046181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:33.046181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:33.046181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:33.046181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:33.046181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:33.046181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:33.046181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:33.046181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:33.047033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:33.047033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:33.047033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:33.047033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:33.047033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:33.047033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:33.047033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:33.047033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:33.047033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:33.047033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:33.047033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:33.047033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:33.047033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:33.047033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:33.047033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:33.047033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:33.047033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:33.048768 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:33.048768 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:33.048768 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:33.048768 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:33.048768 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:33.048768 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:33.048768 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:33.048768 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:33.048768 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 363/705] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:33.049576 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffc34e-8055-4713-bc90-f20203088db3 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:33.049576 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffc34e-8055-4713-bc90-f20203088db3 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:33.049576 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffc34e-8055-4713-bc90-f20203088db3 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:33.051106 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d14e5e-67ba-4af6-a5ea-d658b1683a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:33.055821 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 364/706] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:33.055821 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffc34e-8055-4713-bc90-f20203088db3 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:33.056581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d14e5e-67ba-4af6-a5ea-d658b1683a6f None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:33.058774 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d14e5e-67ba-4af6-a5ea-d658b1683a6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:33.059756 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d14e5e-67ba-4af6-a5ea-d658b1683a6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:33.062114 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d14e5e-67ba-4af6-a5ea-d658b1683a6f None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:33.079697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 343/707] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:33.082053 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac213f1-af04-41ee-8b7e-e0830712197c None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:33.082679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac213f1-af04-41ee-8b7e-e0830712197c None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:33.083384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac213f1-af04-41ee-8b7e-e0830712197c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:33.106134 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-bac213f1-af04-41ee-8b7e-e0830712197c None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:33.108196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-bac213f1-af04-41ee-8b7e-e0830712197c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:33.152179 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:33.168341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffc34e-8055-4713-bc90-f20203088db3 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:33.194082 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 344/708] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:33.194926 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac213f1-af04-41ee-8b7e-e0830712197c None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:33.204629 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4ba95-f787-45d9-96ca-c2d8b4c45e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:33.205340 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4ba95-f787-45d9-96ca-c2d8b4c45e51 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:33.205664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4ba95-f787-45d9-96ca-c2d8b4c45e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:33.205934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c4ba95-f787-45d9-96ca-c2d8b4c45e51 None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:33.206757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4ba95-f787-45d9-96ca-c2d8b4c45e51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:33.207061 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4ba95-f787-45d9-96ca-c2d8b4c45e51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:33.208688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c4ba95-f787-45d9-96ca-c2d8b4c45e51 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:33.223497 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 365/709] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:33.228880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4ba95-f787-45d9-96ca-c2d8b4c45e51 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:33.268835 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:33.346039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c4ba95-f787-45d9-96ca-c2d8b4c45e51 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:33.392186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33465cab-6cde-492e-917f-3600d13e4cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:33.392813 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33465cab-6cde-492e-917f-3600d13e4cdd None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:33.393091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33465cab-6cde-492e-917f-3600d13e4cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:33.393362 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33465cab-6cde-492e-917f-3600d13e4cdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:33.399130 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33465cab-6cde-492e-917f-3600d13e4cdd None admin] RBAC: Adding query filter params (role.id=637ed3cd3d484839ab21de02c2bbe293, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=92318948dcf1407b9fc8aa5c199787ae) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:33.399130 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33465cab-6cde-492e-917f-3600d13e4cdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:33.399130 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33465cab-6cde-492e-917f-3600d13e4cdd None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:33.404063 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 366/710] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:56:33 2022] GET /identity/v3/role_assignments?user.id=92318948dcf1407b9fc8aa5c199787ae&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=637ed3cd3d484839ab21de02c2bbe293 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:33.423505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac213f1-af04-41ee-8b7e-e0830712197c None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:33.473583 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb9618d-bd66-475a-9979-0a6e108ae1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:33.474609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9618d-bd66-475a-9979-0a6e108ae1ce None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:33.475112 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9618d-bd66-475a-9979-0a6e108ae1ce None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:33.475591 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9618d-bd66-475a-9979-0a6e108ae1ce None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:33.476442 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb9618d-bd66-475a-9979-0a6e108ae1ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:33.478127 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb9618d-bd66-475a-9979-0a6e108ae1ce None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:33.517919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 345/711] 10.222.0.36 () {60 vars in 1132 bytes} [Fri Sep 2 08:56:33 2022] POST /identity/v3/users => generated 254 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:56:33.535743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33465cab-6cde-492e-917f-3600d13e4cdd None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:33.650585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:33.653782 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33465cab-6cde-492e-917f-3600d13e4cdd None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:33.691401 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1bc79-8d96-4882-80ff-806f1eb589b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:33.692052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1bc79-8d96-4882-80ff-806f1eb589b3 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:33.692331 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1bc79-8d96-4882-80ff-806f1eb589b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:33.692602 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1bc79-8d96-4882-80ff-806f1eb589b3 None admin] PATH_INFO: `/v3/services/network` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:33.693197 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1bc79-8d96-4882-80ff-806f1eb589b3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:33.694591 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1bc79-8d96-4882-80ff-806f1eb589b3 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:33.707433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-cdb1bc79-8d96-4882-80ff-806f1eb589b3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 08:56:33.707433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:33.707433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:56:33.707433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:56:33.707433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:56:33.707433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:56:33.707433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:56:33.707433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:33.707433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 08:56:33.707433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:33.707433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:56:33.707433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:33.707433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:33.707433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:33.707433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:33.707433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:33.707433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:33.709600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:33.709600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:33.709600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:33.709600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:33.709600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:33.709600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:33.709600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:56:33.709600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:56:33.709600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:56:33.709600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:56:33.709600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:33.709600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:33.709600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:33.709600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:33.709600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:33.709600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:33.711268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:33.711268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:33.711268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:33.711268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:33.711268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:33.711268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:33.711268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:33.711268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:33.711268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:33.711268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:33.711268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:56:33.711268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:33.711268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 08:56:33.711268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:33.711268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 367/712] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:33 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:33.717539 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb9618d-bd66-475a-9979-0a6e108ae1ce None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:33.754038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b9d40-18ba-4797-a401-d0c8e0e1c4c9 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:33.754341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b9d40-18ba-4797-a401-d0c8e0e1c4c9 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:33.754591 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b9d40-18ba-4797-a401-d0c8e0e1c4c9 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:33.756297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 368/713] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:33.758932 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:33.762135 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49703cb-8d18-42b5-822f-4cfb30779f4d None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:33.762606 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49703cb-8d18-42b5-822f-4cfb30779f4d None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:33.762894 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49703cb-8d18-42b5-822f-4cfb30779f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:33.783872 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-a49703cb-8d18-42b5-822f-4cfb30779f4d None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:33.785853 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a49703cb-8d18-42b5-822f-4cfb30779f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:33.861991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb9618d-bd66-475a-9979-0a6e108ae1ce None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:33.872608 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 369/714] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:33.905284 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8892870-0c49-4567-a2eb-9f942d0ca756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:33.906244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8892870-0c49-4567-a2eb-9f942d0ca756 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:33.906718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8892870-0c49-4567-a2eb-9f942d0ca756 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:33.907218 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8892870-0c49-4567-a2eb-9f942d0ca756 None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:33.908042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8892870-0c49-4567-a2eb-9f942d0ca756 None admin] RBAC: Adding query filter params (name=network) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:33.908551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8892870-0c49-4567-a2eb-9f942d0ca756 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:33.910180 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8892870-0c49-4567-a2eb-9f942d0ca756 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:33.915301 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 346/715] 10.222.0.36 () {56 vars in 1112 bytes} [Fri Sep 2 08:56:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:33.921077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49703cb-8d18-42b5-822f-4cfb30779f4d None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:34.020024 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:34.027279 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49703cb-8d18-42b5-822f-4cfb30779f4d None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:34.063818 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad174884-24a5-4e20-854e-7a26cb888e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:34.064776 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad174884-24a5-4e20-854e-7a26cb888e4b None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:34.065251 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad174884-24a5-4e20-854e-7a26cb888e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:34.066377 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad174884-24a5-4e20-854e-7a26cb888e4b None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:34.067264 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad174884-24a5-4e20-854e-7a26cb888e4b None admin] RBAC: Adding query filter params (type=network) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:34.067783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad174884-24a5-4e20-854e-7a26cb888e4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:34.069364 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad174884-24a5-4e20-854e-7a26cb888e4b None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:34.075759 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 370/716] 10.222.0.36 () {56 vars in 1112 bytes} [Fri Sep 2 08:56:33 2022] GET /identity/v3/services?type=network => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:34.287970 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8892870-0c49-4567-a2eb-9f942d0ca756 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:34.390996 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:34.411823 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8892870-0c49-4567-a2eb-9f942d0ca756 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:34.454990 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8a26b-e3b9-497a-9d46-92b2aa00d796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:34.455661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8a26b-e3b9-497a-9d46-92b2aa00d796 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:34.455986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8a26b-e3b9-497a-9d46-92b2aa00d796 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:34.456254 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8a26b-e3b9-497a-9d46-92b2aa00d796 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:34.456833 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8a26b-e3b9-497a-9d46-92b2aa00d796 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:34.458313 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8a26b-e3b9-497a-9d46-92b2aa00d796 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:34.474430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-19e8a26b-e3b9-497a-9d46-92b2aa00d796 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 08:56:34.474430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:34.474430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:56:34.474430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:56:34.474430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:56:34.474430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:56:34.474430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:56:34.474430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:34.474430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 08:56:34.474430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:34.474430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:56:34.474430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:34.474430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:34.474430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:34.474430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:34.474430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:34.474430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:34.475501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:34.475501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:34.475501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:34.475501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:34.475501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:34.475501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:34.475501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:56:34.475501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:56:34.475501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:56:34.475501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:56:34.475501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:34.475501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:34.475501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:34.475501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:34.475501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:34.475501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:34.476334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:34.476334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:34.476334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:34.476334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:34.476334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:34.476334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:34.476334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:34.476334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:34.476334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:34.476334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:34.476334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:56:34.476334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:34.476334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 08:56:34.476334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:34.476334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 347/717] 10.222.0.36 () {56 vars in 1129 bytes} [Fri Sep 2 08:56:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:34.483391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad174884-24a5-4e20-854e-7a26cb888e4b None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:34.645902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad174884-24a5-4e20-854e-7a26cb888e4b None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:34.688085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bbffa8-c23a-46fb-899d-ff3a9898c915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:34.689084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbffa8-c23a-46fb-899d-ff3a9898c915 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:34.689560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbffa8-c23a-46fb-899d-ff3a9898c915 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:34.690022 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bbffa8-c23a-46fb-899d-ff3a9898c915 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:34.690900 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbffa8-c23a-46fb-899d-ff3a9898c915 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:34.691437 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbffa8-c23a-46fb-899d-ff3a9898c915 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:34.692998 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bbffa8-c23a-46fb-899d-ff3a9898c915 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:34.697949 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 371/718] 10.222.0.36 () {56 vars in 1124 bytes} [Fri Sep 2 08:56:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:34.705661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8a26b-e3b9-497a-9d46-92b2aa00d796 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:34.856835 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8a26b-e3b9-497a-9d46-92b2aa00d796 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:34.903531 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be39488-faa7-46f1-9123-20c29c5ce1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:34.904508 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be39488-faa7-46f1-9123-20c29c5ce1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:34.904990 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be39488-faa7-46f1-9123-20c29c5ce1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:34.905693 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be39488-faa7-46f1-9123-20c29c5ce1d4 None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:34.906571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be39488-faa7-46f1-9123-20c29c5ce1d4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:34.907104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be39488-faa7-46f1-9123-20c29c5ce1d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:34.908673 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be39488-faa7-46f1-9123-20c29c5ce1d4 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:34.915868 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 348/719] 10.222.0.36 () {56 vars in 1124 bytes} [Fri Sep 2 08:56:34 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:35.052021 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4687e2-f94f-4c26-b71b-368f941889fd None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:35.052673 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4687e2-f94f-4c26-b71b-368f941889fd None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:35.053166 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4687e2-f94f-4c26-b71b-368f941889fd None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:35.055111 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 372/720] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:35.060911 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d737c1c-9ef3-46d8-8141-639d3acd40ee None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:35.061517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d737c1c-9ef3-46d8-8141-639d3acd40ee None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:35.062861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d737c1c-9ef3-46d8-8141-639d3acd40ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:35.093374 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-5d737c1c-9ef3-46d8-8141-639d3acd40ee None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:35.094985 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-5d737c1c-9ef3-46d8-8141-639d3acd40ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:35.117766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:35.185523 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 349/721] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:35.524051 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4687e2-f94f-4c26-b71b-368f941889fd None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:35.557811 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:35.641430 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4687e2-f94f-4c26-b71b-368f941889fd None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:35.675798 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1c062-b29b-46f2-8872-2bc86c8f321e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:35.676432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1c062-b29b-46f2-8872-2bc86c8f321e None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:35.676701 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1c062-b29b-46f2-8872-2bc86c8f321e None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:35.677024 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1c062-b29b-46f2-8872-2bc86c8f321e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:35.688034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da1c062-b29b-46f2-8872-2bc86c8f321e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:35.693806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-6da1c062-b29b-46f2-8872-2bc86c8f321e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:56:35.693806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:35.693806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:35.693806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:35.693806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:35.693806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:35.693806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:35.693806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:35.693806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:35.693806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:35.693806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:35.693806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:35.693806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:35.693806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:35.693806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:35.693806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:35.694801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:35.694801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:35.694801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:35.694801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:35.694801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:35.694801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:35.694801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:35.694801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:35.694801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:35.694801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:35.694801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:35.694801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:35.694801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:35.694801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:35.694801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:35.694801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:35.695731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:35.695731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:35.695731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:35.695731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:35.695731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:35.695731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:35.695731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:35.695731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:35.695731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:35.695731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:35.695731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:56:35.695731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:35.695731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 373/722] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:56:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:35.704158 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d737c1c-9ef3-46d8-8141-639d3acd40ee None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:35.845796 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d737c1c-9ef3-46d8-8141-639d3acd40ee None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:35.881423 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec74e5-d9ac-48b8-b3e9-5c61e3d9c82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:35.882043 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec74e5-d9ac-48b8-b3e9-5c61e3d9c82a None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:35.882325 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec74e5-d9ac-48b8-b3e9-5c61e3d9c82a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:35.882600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ec74e5-d9ac-48b8-b3e9-5c61e3d9c82a None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:35.883353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec74e5-d9ac-48b8-b3e9-5c61e3d9c82a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:35.883663 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec74e5-d9ac-48b8-b3e9-5c61e3d9c82a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:35.885045 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ec74e5-d9ac-48b8-b3e9-5c61e3d9c82a None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:35.893707 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 350/723] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:56:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:35.900202 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1c062-b29b-46f2-8872-2bc86c8f321e None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:36.015904 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:36.026933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1c062-b29b-46f2-8872-2bc86c8f321e None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:36.062683 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ffb56-1117-41d1-b090-8fe27ac9b4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:36.063341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ffb56-1117-41d1-b090-8fe27ac9b4ae None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:36.063615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ffb56-1117-41d1-b090-8fe27ac9b4ae None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:36.063883 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ffb56-1117-41d1-b090-8fe27ac9b4ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:36.075033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ffb56-1117-41d1-b090-8fe27ac9b4ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:36.076628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ffb56-1117-41d1-b090-8fe27ac9b4ae None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:36.091404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-b93ffb56-1117-41d1-b090-8fe27ac9b4ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:36.091404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:36.091404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:36.091404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:36.091404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:36.091404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:36.091404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:36.091404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:36.091404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:36.091404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:36.091404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:36.091404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:36.091404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:36.091404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:36.091404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:36.091404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:36.092387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:36.092387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:36.092387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:36.092387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:36.092387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:36.092387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:36.092387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:36.092387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:36.092387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:36.092387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:36.092387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:36.092387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:36.092387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:36.092387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:36.092387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:36.092387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:36.092387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:36.093269 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:36.093269 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:36.093269 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:36.093269 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 374/724] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:36.100209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec74e5-d9ac-48b8-b3e9-5c61e3d9c82a None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:36.116790 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:36.173040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52444800-f6e9-4abc-bca1-032f72750e9e None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:36.173698 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52444800-f6e9-4abc-bca1-032f72750e9e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:36.174178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52444800-f6e9-4abc-bca1-032f72750e9e None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:36.176639 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 375/725] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:56:36.182092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4c15a-21d3-4ebd-9268-dd8f36afc265 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:36.182602 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4c15a-21d3-4ebd-9268-dd8f36afc265 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:36.183115 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4c15a-21d3-4ebd-9268-dd8f36afc265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:36.203203 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-86c4c15a-21d3-4ebd-9268-dd8f36afc265 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:36.205126 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-86c4c15a-21d3-4ebd-9268-dd8f36afc265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:36.250138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ec74e5-d9ac-48b8-b3e9-5c61e3d9c82a None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:36.286700 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2dfd0-a5e3-484f-9a54-47f5180d1e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:36.287707 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2dfd0-a5e3-484f-9a54-47f5180d1e6e None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:36.288201 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2dfd0-a5e3-484f-9a54-47f5180d1e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:36.288741 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2dfd0-a5e3-484f-9a54-47f5180d1e6e None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:36.289690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2dfd0-a5e3-484f-9a54-47f5180d1e6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:36.290185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2dfd0-a5e3-484f-9a54-47f5180d1e6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:36.291883 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2dfd0-a5e3-484f-9a54-47f5180d1e6e None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:36.306241 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 351/726] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:36.307733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d3421-2fcd-40a6-aa4e-77489c7b5f80 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:36.308210 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d3421-2fcd-40a6-aa4e-77489c7b5f80 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:36.308673 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d3421-2fcd-40a6-aa4e-77489c7b5f80 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:36.310542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 352/727] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:36.312998 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d3421-2fcd-40a6-aa4e-77489c7b5f80 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:36.319348 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 376/728] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 08:56:36.326381 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdaab2b-75c3-45be-8650-a7e0192d8c30 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:36.326381 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdaab2b-75c3-45be-8650-a7e0192d8c30 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:36.326381 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdaab2b-75c3-45be-8650-a7e0192d8c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:36.343093 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-fbdaab2b-75c3-45be-8650-a7e0192d8c30 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:36.343093 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-fbdaab2b-75c3-45be-8650-a7e0192d8c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:36.417616 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:36.422961 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:36.433065 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 377/729] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:36.434567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68585f21-9fd0-433b-8ac8-9037d1fdb8e1 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:36.434851 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68585f21-9fd0-433b-8ac8-9037d1fdb8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:36.443194 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68585f21-9fd0-433b-8ac8-9037d1fdb8e1 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:36.448537 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 378/730] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:36 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:36.449909 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e954f6-650e-4a97-8973-fce9b468bc02 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:36.450199 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e954f6-650e-4a97-8973-fce9b468bc02 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:36.450478 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e954f6-650e-4a97-8973-fce9b468bc02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:36.462336 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d3421-2fcd-40a6-aa4e-77489c7b5f80 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:36.491163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-d1e954f6-650e-4a97-8973-fce9b468bc02 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:36.492355 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-d1e954f6-650e-4a97-8973-fce9b468bc02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:36.501749 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33202721-c71b-440b-84c0-6c52d523ecfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:36.502636 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33202721-c71b-440b-84c0-6c52d523ecfc None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:36.503357 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33202721-c71b-440b-84c0-6c52d523ecfc None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:36.503834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33202721-c71b-440b-84c0-6c52d523ecfc None admin] PATH_INFO: `/v3/users/nova` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:36.522554 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33202721-c71b-440b-84c0-6c52d523ecfc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:36.526179 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33202721-c71b-440b-84c0-6c52d523ecfc None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:36.553623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-33202721-c71b-440b-84c0-6c52d523ecfc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 08:56:36.553623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:36.553623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:36.553623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:36.553623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:36.553623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:36.553623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:36.553623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:36.553623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:36.553623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:36.553623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:36.553623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:36.553623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:36.553623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:36.553623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:36.553623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:36.554630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:36.554630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:36.554630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:36.554630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:36.554630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:36.554630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:36.554630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:36.554630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:36.554630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:36.554630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:36.554630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:36.554630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:36.554630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:36.554630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:36.554630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:36.554630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:36.554630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:36.554630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:36.555677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:36.555677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:36.555677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:36.555677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:36.555677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:36.555677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:36.555677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:36.555677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:36.555677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 08:56:36.555677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:36.560526 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 353/731] 10.222.0.36 () {56 vars in 1093 bytes} [Fri Sep 2 08:56:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:36.567208 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33202721-c71b-440b-84c0-6c52d523ecfc None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:36.618220 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 379/732] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:36.648862 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e954f6-650e-4a97-8973-fce9b468bc02 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:36.687782 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33202721-c71b-440b-84c0-6c52d523ecfc None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:36.717283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:36.722874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc91f8f-659c-4e79-8165-cb6826e76d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:36.723512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc91f8f-659c-4e79-8165-cb6826e76d99 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:36.723794 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc91f8f-659c-4e79-8165-cb6826e76d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:36.724067 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc91f8f-659c-4e79-8165-cb6826e76d99 None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:36.724787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc91f8f-659c-4e79-8165-cb6826e76d99 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:36.725082 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc91f8f-659c-4e79-8165-cb6826e76d99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:36.726505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc91f8f-659c-4e79-8165-cb6826e76d99 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:36.743621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 354/733] 10.222.0.36 () {56 vars in 1133 bytes} [Fri Sep 2 08:56:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:36.745409 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc91f8f-659c-4e79-8165-cb6826e76d99 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:36.797318 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e954f6-650e-4a97-8973-fce9b468bc02 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:36.826989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:36.840656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed126b6d-f80f-46f4-b373-54380fa96ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:36.841648 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed126b6d-f80f-46f4-b373-54380fa96ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:36.842076 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed126b6d-f80f-46f4-b373-54380fa96ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:36.842496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed126b6d-f80f-46f4-b373-54380fa96ee5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:36.854414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed126b6d-f80f-46f4-b373-54380fa96ee5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:36.859756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-ed126b6d-f80f-46f4-b373-54380fa96ee5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:56:36.859756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:36.859756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:36.859756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:36.859756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:36.859756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:36.859756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:36.859756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:36.859756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:36.859756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:36.859756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:36.859756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:36.859756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:36.859756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:36.859756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:36.859756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:36.860910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:36.860910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:36.860910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:36.860910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:36.860910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:36.860910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:36.860910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:36.860910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:36.860910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:36.860910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:36.860910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:36.860910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:36.860910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:36.860910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:36.860910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:36.860910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:36.861815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:36.861815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:36.861815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:36.861815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:36.861815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:36.861815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:36.861815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:36.861815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:36.861815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:36.861815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:36.861815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:56:36.861815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:36.864187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 380/734] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:56:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:36.865161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed126b6d-f80f-46f4-b373-54380fa96ee5 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:36.866787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc91f8f-659c-4e79-8165-cb6826e76d99 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:36.902693 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f4c3f-07a1-4400-9d8a-37120cb5e492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:36.903226 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f4c3f-07a1-4400-9d8a-37120cb5e492 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:36.903503 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f4c3f-07a1-4400-9d8a-37120cb5e492 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:36.903771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f4c3f-07a1-4400-9d8a-37120cb5e492 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:36.914174 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f4c3f-07a1-4400-9d8a-37120cb5e492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:36.919274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-862f4c3f-07a1-4400-9d8a-37120cb5e492 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:36.919274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:36.919274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:36.919274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:36.919274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:36.919274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:36.919274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:36.919274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:36.919274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:36.919274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:36.919274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:36.919274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:36.919274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:36.919274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:36.919274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:36.919274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:36.920488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:36.920488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:36.920488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:36.920488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:36.920488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:36.920488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:36.920488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:36.920488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:36.920488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:36.920488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:36.920488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:36.920488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:36.920488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:36.920488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:36.920488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:36.920488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:36.921388 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:36.921388 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:36.921388 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:36.921388 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:36.921388 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:36.921388 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:36.921388 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:36.921388 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:36.921388 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:36.921388 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:36.921388 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:36.921388 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:36.921388 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 355/735] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:56:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:36.921388 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f4c3f-07a1-4400-9d8a-37120cb5e492 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:36.980125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:36.987397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed126b6d-f80f-46f4-b373-54380fa96ee5 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:37.023391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f4406-1454-4bf5-955b-b0da29344b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:37.024013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f4406-1454-4bf5-955b-b0da29344b5c None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:37.024315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f4406-1454-4bf5-955b-b0da29344b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:37.024557 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f4406-1454-4bf5-955b-b0da29344b5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:37.034650 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f4406-1454-4bf5-955b-b0da29344b5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:37.036119 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f4406-1454-4bf5-955b-b0da29344b5c None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:37.043958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f4c3f-07a1-4400-9d8a-37120cb5e492 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:37.051820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-017f4406-1454-4bf5-955b-b0da29344b5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:37.051820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:37.051820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:37.051820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:37.051820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:37.051820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:37.051820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:37.051820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:37.051820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:37.051820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:37.051820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:37.051820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:37.051820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:37.051820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:37.051820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:37.051820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:37.052840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:37.052840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:37.052840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:37.052840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:37.052840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:37.052840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:37.052840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:37.052840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:37.052840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:37.052840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:37.052840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:37.052840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:37.052840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:37.052840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:37.052840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:37.052840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:37.052840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:37.058774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:37.058774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:37.058774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:37.058774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 381/736] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:37.058774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f4406-1454-4bf5-955b-b0da29344b5c None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:37.087775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b068a-be16-4dc5-83ce-c595e0567458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:37.088410 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b068a-be16-4dc5-83ce-c595e0567458 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:37.088686 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b068a-be16-4dc5-83ce-c595e0567458 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:37.088953 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725b068a-be16-4dc5-83ce-c595e0567458 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:37.089610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b068a-be16-4dc5-83ce-c595e0567458 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:37.089906 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b068a-be16-4dc5-83ce-c595e0567458 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:37.091286 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725b068a-be16-4dc5-83ce-c595e0567458 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:37.094900 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:37.099988 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 356/737] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:56:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:37.100684 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b068a-be16-4dc5-83ce-c595e0567458 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:37.174219 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f4406-1454-4bf5-955b-b0da29344b5c None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:37.210468 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4f7a4-8a29-4c63-8596-38ccfde499ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:37.210669 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725b068a-be16-4dc5-83ce-c595e0567458 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:37.211091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4f7a4-8a29-4c63-8596-38ccfde499ea None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:37.211384 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4f7a4-8a29-4c63-8596-38ccfde499ea None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:37.211660 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4f7a4-8a29-4c63-8596-38ccfde499ea None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:37.212327 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4f7a4-8a29-4c63-8596-38ccfde499ea None admin] RBAC: Adding query filter params (name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:37.212631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4f7a4-8a29-4c63-8596-38ccfde499ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:37.214099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4f7a4-8a29-4c63-8596-38ccfde499ea None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:37.218114 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:37.227023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 382/738] 10.222.0.36 () {56 vars in 1103 bytes} [Fri Sep 2 08:56:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:37.227023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4f7a4-8a29-4c63-8596-38ccfde499ea None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:37.250981 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b87075-353c-4fab-bfdb-666b433d5d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:37.251617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b87075-353c-4fab-bfdb-666b433d5d3c None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:37.252084 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b87075-353c-4fab-bfdb-666b433d5d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:37.252194 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b87075-353c-4fab-bfdb-666b433d5d3c None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:37.253410 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b87075-353c-4fab-bfdb-666b433d5d3c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:37.254720 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b87075-353c-4fab-bfdb-666b433d5d3c None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:37.270829 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 357/739] 10.222.0.36 () {60 vars in 1141 bytes} [Fri Sep 2 08:56:37 2022] POST /identity/v3/services => generated 240 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:56:37.271687 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b87075-353c-4fab-bfdb-666b433d5d3c None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:37.365834 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:37.371270 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4f7a4-8a29-4c63-8596-38ccfde499ea None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:37.402213 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b87075-353c-4fab-bfdb-666b433d5d3c None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:37.414753 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a659a0-1540-4dcb-b9a8-104a2885fdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:37.415779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a659a0-1540-4dcb-b9a8-104a2885fdd6 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:37.416263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a659a0-1540-4dcb-b9a8-104a2885fdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:37.416751 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a659a0-1540-4dcb-b9a8-104a2885fdd6 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:37.417699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a659a0-1540-4dcb-b9a8-104a2885fdd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:37.418253 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a659a0-1540-4dcb-b9a8-104a2885fdd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:37.419783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a659a0-1540-4dcb-b9a8-104a2885fdd6 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:37.434864 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 383/740] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:37.435625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a659a0-1540-4dcb-b9a8-104a2885fdd6 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:37.442260 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68739b98-3c60-41a6-8600-2ca294612f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:37.442680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68739b98-3c60-41a6-8600-2ca294612f20 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:37.442963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68739b98-3c60-41a6-8600-2ca294612f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:37.443292 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68739b98-3c60-41a6-8600-2ca294612f20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:37.454426 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68739b98-3c60-41a6-8600-2ca294612f20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:37.456042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68739b98-3c60-41a6-8600-2ca294612f20 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:37.472560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-68739b98-3c60-41a6-8600-2ca294612f20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:37.472560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:37.472560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:37.472560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:37.472560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:37.472560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:37.472560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:37.472560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:37.472560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:37.472560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:37.472560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:37.472560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:37.472560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:37.472560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:37.472560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:37.472560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:37.473522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:37.473522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:37.473522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:37.473522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:37.473522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:37.473522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:37.473522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:37.473522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:37.473522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:37.473522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:37.473522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:37.473522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:37.473522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:37.473522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:37.473522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:37.473522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:37.473522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:37.474413 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:37.474413 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:37.474413 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:37.474413 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 358/741] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:37.475313 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68739b98-3c60-41a6-8600-2ca294612f20 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:37.492853 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:37.553199 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a659a0-1540-4dcb-b9a8-104a2885fdd6 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:37.580419 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68739b98-3c60-41a6-8600-2ca294612f20 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:37.589122 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8a012-3155-43d0-9424-291b64dac14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:37.589753 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8a012-3155-43d0-9424-291b64dac14c None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:37.590038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8a012-3155-43d0-9424-291b64dac14c None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:37.590310 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8a012-3155-43d0-9424-291b64dac14c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:37.600847 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e8a012-3155-43d0-9424-291b64dac14c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:37.602295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e8a012-3155-43d0-9424-291b64dac14c None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:37.616784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9078d1d-b411-4147-bb18-575866304ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:37.617666 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9078d1d-b411-4147-bb18-575866304ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:37.618114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-35e8a012-3155-43d0-9424-291b64dac14c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:37.618114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:37.618114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:37.618114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:37.618114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:37.618114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:37.618114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:37.618114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:37.618114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:37.618114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:37.618114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:37.618114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:37.618114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:37.618114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:37.618114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:37.618114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:37.619789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:37.619789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:37.619789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:37.619789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:37.619789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:37.619789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:37.619789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:37.619789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:37.619789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:37.619789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:37.619789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:37.619789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:37.619789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:37.619789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:37.619789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:37.619789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:37.619789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:37.620952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:37.620952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:37.620952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:37.620952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 384/742] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:37.620952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2d95f-1748-4c9b-96ea-1f10a42001ff None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:37.620952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2d95f-1748-4c9b-96ea-1f10a42001ff None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:37.621298 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2d95f-1748-4c9b-96ea-1f10a42001ff None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:37.621617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9078d1d-b411-4147-bb18-575866304ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:37.622099 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9078d1d-b411-4147-bb18-575866304ce0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:37.622476 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 385/743] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:37.623244 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e2d95f-1748-4c9b-96ea-1f10a42001ff None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:37.639557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9078d1d-b411-4147-bb18-575866304ce0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:37.639831 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:37.641177 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9078d1d-b411-4147-bb18-575866304ce0 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:37.653233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-f9078d1d-b411-4147-bb18-575866304ce0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:37.653233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:37.653233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:37.653233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:37.653233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:37.653233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:37.653233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:37.653233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:37.653233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:37.653233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:37.653233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:37.653233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:37.653233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:37.653233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:37.653233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:37.654243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:37.654243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:37.654243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:37.654243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:37.654243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:37.654243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:37.654243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:37.654243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:37.654243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:37.654243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:37.654243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:37.654243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:37.654243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:37.654243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:37.654243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:37.654243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:37.654243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:37.655209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:37.655209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:37.655209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:37.655209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:37.655209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:37.655209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:37.655209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:37.655209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:37.655209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 359/744] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:37.655209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9078d1d-b411-4147-bb18-575866304ce0 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:37.734268 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:37.747566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e2d95f-1748-4c9b-96ea-1f10a42001ff None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:37.762889 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9078d1d-b411-4147-bb18-575866304ce0 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:37.786300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd250bd-78fe-425f-ad66-45c36d0df1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:37.786906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd250bd-78fe-425f-ad66-45c36d0df1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:37.787247 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd250bd-78fe-425f-ad66-45c36d0df1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:37.787487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd250bd-78fe-425f-ad66-45c36d0df1f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:37.788121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd250bd-78fe-425f-ad66-45c36d0df1f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:37.788420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd250bd-78fe-425f-ad66-45c36d0df1f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:37.789781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd250bd-78fe-425f-ad66-45c36d0df1f9 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:37.799168 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a31ea-5823-4e33-9a6e-120358c977c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:37.800373 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a31ea-5823-4e33-9a6e-120358c977c4 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:37.800656 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a31ea-5823-4e33-9a6e-120358c977c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:37.800931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a31ea-5823-4e33-9a6e-120358c977c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:37.801580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a31ea-5823-4e33-9a6e-120358c977c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:37.801885 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a31ea-5823-4e33-9a6e-120358c977c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:37.803271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a31ea-5823-4e33-9a6e-120358c977c4 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:37.808102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 386/745] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:37.808102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee5b9c-f726-47b5-ad62-9d529edeb0fd None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:37.808102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee5b9c-f726-47b5-ad62-9d529edeb0fd None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:37.808102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee5b9c-f726-47b5-ad62-9d529edeb0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:37.824479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 360/746] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:37.824479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a31ea-5823-4e33-9a6e-120358c977c4 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:37.824717 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-0dee5b9c-f726-47b5-ad62-9d529edeb0fd None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:37.824717 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0dee5b9c-f726-47b5-ad62-9d529edeb0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:37.891116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:37.903111 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 387/747] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:37 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 08:56:37.903111 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee5b9c-f726-47b5-ad62-9d529edeb0fd None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:37.923335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a31ea-5823-4e33-9a6e-120358c977c4 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:37.963631 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c5639-1b63-4e8e-b787-29037249fe7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:37.964346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c5639-1b63-4e8e-b787-29037249fe7a None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:37.964636 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c5639-1b63-4e8e-b787-29037249fe7a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:37.964912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c5639-1b63-4e8e-b787-29037249fe7a None admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:37.965609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948c5639-1b63-4e8e-b787-29037249fe7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:37.966020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948c5639-1b63-4e8e-b787-29037249fe7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:37.969028 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948c5639-1b63-4e8e-b787-29037249fe7a None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:37.970681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:37.984125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 361/748] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:37.985088 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c5639-1b63-4e8e-b787-29037249fe7a None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:38.046641 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee5b9c-f726-47b5-ad62-9d529edeb0fd None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:38.093891 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b200b70-634a-4436-835c-11194214ed90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:38.094671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b200b70-634a-4436-835c-11194214ed90 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:38.095034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b200b70-634a-4436-835c-11194214ed90 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:38.095430 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b200b70-634a-4436-835c-11194214ed90 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:38.095602 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c5639-1b63-4e8e-b787-29037249fe7a None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:38.109481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b200b70-634a-4436-835c-11194214ed90 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:38.111908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b200b70-634a-4436-835c-11194214ed90 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:38.128213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-1b200b70-634a-4436-835c-11194214ed90 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 02 08:56:38.128213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:38.128213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:38.128213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:38.128213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:38.128213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:38.128213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:38.128213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:38.128213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:38.128213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:38.128213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:38.128213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:38.128213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:38.128213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:38.128213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:38.128213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:38.129300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:38.129300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:38.129300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:38.129300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:38.129300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:38.129300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:38.129300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:38.129300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:38.129300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:38.129300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:38.129300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:38.129300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:38.129300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:38.129300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:38.129300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:38.129300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:38.129300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:38.129300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:38.130508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:38.130508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:38.130508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:38.130508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:38.130508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:38.130508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:38.130508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:38.130508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:38.130508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 02 08:56:38.130508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:38.131200 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 388/749] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:56:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:38.131831 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b200b70-634a-4436-835c-11194214ed90 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:38.135536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea061491-785f-4ca8-9f36-b0eb8279233b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:38.136454 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea061491-785f-4ca8-9f36-b0eb8279233b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:38.136831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea061491-785f-4ca8-9f36-b0eb8279233b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:38.137205 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea061491-785f-4ca8-9f36-b0eb8279233b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:38.148914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea061491-785f-4ca8-9f36-b0eb8279233b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:38.150554 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea061491-785f-4ca8-9f36-b0eb8279233b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:38.164015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-ea061491-785f-4ca8-9f36-b0eb8279233b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 08:56:38.164015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:38.164015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:38.164015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:38.164015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:38.164015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:38.164015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:38.164015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:38.164015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:38.164015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:38.164015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:38.164015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:38.164015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:38.164015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:38.164015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:38.164015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:38.165186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:38.165186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:38.165186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:38.165186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:38.165186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:38.165186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:38.165186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:38.165186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:38.165186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:38.165186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:38.165186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:38.165186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:38.165186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:38.165186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:38.165186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:38.165186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:38.165186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:38.165186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:38.166363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:38.166363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:38.166363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:38.166363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:38.166363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:38.166363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:38.166363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:38.166363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:38.166363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 08:56:38.166363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:38.172494 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 362/750] 10.222.0.36 () {56 vars in 1117 bytes} [Fri Sep 2 08:56:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:38.172494 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea061491-785f-4ca8-9f36-b0eb8279233b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:38.220174 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:38.229576 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:38.266468 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b200b70-634a-4436-835c-11194214ed90 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:38.273545 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea061491-785f-4ca8-9f36-b0eb8279233b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:38.301581 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b608db1-abb7-458d-8ae6-18d97526bee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:38.302228 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b608db1-abb7-458d-8ae6-18d97526bee0 None admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:38.302506 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b608db1-abb7-458d-8ae6-18d97526bee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:38.303056 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b608db1-abb7-458d-8ae6-18d97526bee0 None admin] PATH_INFO: `/v3/projects/eef80493db09437b88f2b147a59928c1/users/cce502a83f5343558c2136c0193e7d85/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:38.304982 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b608db1-abb7-458d-8ae6-18d97526bee0 None admin] RBAC: Authorizing `identity:create_grant(project_id=eef80493db09437b88f2b147a59928c1, user_id=cce502a83f5343558c2136c0193e7d85, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:38.306397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b608db1-abb7-458d-8ae6-18d97526bee0 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:38.307698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdce6c7-3fe6-4dc6-97de-6cd7b79302ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:38.308423 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdce6c7-3fe6-4dc6-97de-6cd7b79302ed None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:38.308746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdce6c7-3fe6-4dc6-97de-6cd7b79302ed None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:38.309328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdce6c7-3fe6-4dc6-97de-6cd7b79302ed None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:38.310149 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdce6c7-3fe6-4dc6-97de-6cd7b79302ed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:38.310518 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdce6c7-3fe6-4dc6-97de-6cd7b79302ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:38.312171 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdce6c7-3fe6-4dc6-97de-6cd7b79302ed None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:38.331378 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 389/751] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:56:38 2022] PUT /identity/v3/projects/eef80493db09437b88f2b147a59928c1/users/cce502a83f5343558c2136c0193e7d85/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:56:38.331378 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b608db1-abb7-458d-8ae6-18d97526bee0 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:38.348018 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 363/752] 10.222.0.36 () {56 vars in 1135 bytes} [Fri Sep 2 08:56:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:38.349024 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdce6c7-3fe6-4dc6-97de-6cd7b79302ed None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:38.464694 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdce6c7-3fe6-4dc6-97de-6cd7b79302ed None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:38.480446 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:38.484675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b608db1-abb7-458d-8ae6-18d97526bee0 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:38.510006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d2433-f6d1-4be9-9a57-ca1d9d190bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:38.510626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d2433-f6d1-4be9-9a57-ca1d9d190bda None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:38.510906 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d2433-f6d1-4be9-9a57-ca1d9d190bda None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:38.511211 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d2433-f6d1-4be9-9a57-ca1d9d190bda None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:38.511866 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d2433-f6d1-4be9-9a57-ca1d9d190bda None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:38.513242 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d2433-f6d1-4be9-9a57-ca1d9d190bda None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:38.513924 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:38.530324 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 364/753] 10.222.0.36 () {60 vars in 1141 bytes} [Fri Sep 2 08:56:38 2022] POST /identity/v3/services => generated 251 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:56:38.531653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d2433-f6d1-4be9-9a57-ca1d9d190bda None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:38.544255 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbedf36-d239-4c21-972b-8c7b803faaf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:38.544874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbedf36-d239-4c21-972b-8c7b803faaf4 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:38.545152 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbedf36-d239-4c21-972b-8c7b803faaf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:38.545422 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbedf36-d239-4c21-972b-8c7b803faaf4 None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:38.546223 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbedf36-d239-4c21-972b-8c7b803faaf4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:38.546570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbedf36-d239-4c21-972b-8c7b803faaf4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:38.550286 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbedf36-d239-4c21-972b-8c7b803faaf4 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:38.577031 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 390/754] 10.222.0.36 () {56 vars in 1149 bytes} [Fri Sep 2 08:56:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:38.584083 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbedf36-d239-4c21-972b-8c7b803faaf4 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:38.703885 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d2433-f6d1-4be9-9a57-ca1d9d190bda None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:38.746576 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3927ef7-a6bb-42ae-ab6e-13cce8d719c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:38.747358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3927ef7-a6bb-42ae-ab6e-13cce8d719c0 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:38.747550 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3927ef7-a6bb-42ae-ab6e-13cce8d719c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:38.747823 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3927ef7-a6bb-42ae-ab6e-13cce8d719c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:38.757862 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbedf36-d239-4c21-972b-8c7b803faaf4 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:38.806032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cfe6e1-08d1-4924-9223-40adaf1dc761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:38.806985 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfe6e1-08d1-4924-9223-40adaf1dc761 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:38.807882 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfe6e1-08d1-4924-9223-40adaf1dc761 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:38.808349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cfe6e1-08d1-4924-9223-40adaf1dc761 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:38.810156 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:38.839371 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3927ef7-a6bb-42ae-ab6e-13cce8d719c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:38.840842 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3927ef7-a6bb-42ae-ab6e-13cce8d719c0 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:38.859715 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:38.887640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cfe6e1-08d1-4924-9223-40adaf1dc761 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:38.889559 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cfe6e1-08d1-4924-9223-40adaf1dc761 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:38.975557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 365/755] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:38 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 08:56:38.980862 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3927ef7-a6bb-42ae-ab6e-13cce8d719c0 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:38.998330 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 391/756] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:38 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 02 08:56:39.003948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cfe6e1-08d1-4924-9223-40adaf1dc761 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:39.113022 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cfe6e1-08d1-4924-9223-40adaf1dc761 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:39.123545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:39.152939 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844d338-a37c-45fd-a6b8-7441a1d663a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:39.153938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844d338-a37c-45fd-a6b8-7441a1d663a2 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:39.154414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844d338-a37c-45fd-a6b8-7441a1d663a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:39.154892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b844d338-a37c-45fd-a6b8-7441a1d663a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:39.167004 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844d338-a37c-45fd-a6b8-7441a1d663a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:39.169490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b844d338-a37c-45fd-a6b8-7441a1d663a2 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:39.187720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-b844d338-a37c-45fd-a6b8-7441a1d663a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:39.187720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:39.187720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:39.187720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:39.187720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:39.187720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:39.187720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:39.187720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:39.187720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:39.187720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:39.187720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:39.187720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:39.187720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:39.187720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:39.187720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:39.187720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:39.188739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:39.188739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:39.188739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:39.188739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:39.188739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:39.188739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:39.188739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:39.188739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:39.188739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:39.188739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:39.188739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:39.188739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:39.188739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:39.188739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:39.188739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:39.188739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:39.188739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:39.189635 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:39.189635 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:39.189635 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:39.191464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:39.193552 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 392/757] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:39.198645 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844d338-a37c-45fd-a6b8-7441a1d663a2 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:39.244773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3927ef7-a6bb-42ae-ab6e-13cce8d719c0 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:39.285307 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303bf7f-b1d7-43bf-8c92-d6bdd0049589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:39.285954 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303bf7f-b1d7-43bf-8c92-d6bdd0049589 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:39.286228 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303bf7f-b1d7-43bf-8c92-d6bdd0049589 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:39.286496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303bf7f-b1d7-43bf-8c92-d6bdd0049589 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:39.297430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303bf7f-b1d7-43bf-8c92-d6bdd0049589 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:39.299364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303bf7f-b1d7-43bf-8c92-d6bdd0049589 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:39.315358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-6303bf7f-b1d7-43bf-8c92-d6bdd0049589 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:39.315358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:39.315358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:39.315358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:39.315358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:39.315358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:39.315358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:39.315358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:39.315358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:39.315358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:39.315358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:39.315358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:39.315358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:39.315358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:39.315358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:39.315358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:39.316347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:39.316347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:39.316347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:39.316347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:39.316347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:39.316347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:39.316347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:39.316347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:39.316347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:39.316347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:39.316347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:39.316347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:39.316347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:39.316347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:39.316347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:39.316347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:39.316347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:39.317248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:39.317248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:39.317248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:39.317248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 366/758] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:39.317991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222e4ab-c38a-4610-8dfe-7cd281086cc3 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:39.318317 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222e4ab-c38a-4610-8dfe-7cd281086cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:39.318523 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222e4ab-c38a-4610-8dfe-7cd281086cc3 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:39.323349 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 367/759] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:39.324163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222e4ab-c38a-4610-8dfe-7cd281086cc3 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:39.373022 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b844d338-a37c-45fd-a6b8-7441a1d663a2 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:39.407526 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:39.422835 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65509be0-813b-4a07-b56a-cc79cd0c7226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:39.423564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65509be0-813b-4a07-b56a-cc79cd0c7226 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:39.423913 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65509be0-813b-4a07-b56a-cc79cd0c7226 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:39.424282 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65509be0-813b-4a07-b56a-cc79cd0c7226 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:39.425037 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65509be0-813b-4a07-b56a-cc79cd0c7226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:39.425415 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65509be0-813b-4a07-b56a-cc79cd0c7226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:39.426838 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65509be0-813b-4a07-b56a-cc79cd0c7226 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:39.441560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 393/760] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:39.443398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ffb42-1d0d-4c10-a3e4-48571e98d4a5 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:39.443686 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ffb42-1d0d-4c10-a3e4-48571e98d4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:39.443958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ffb42-1d0d-4c10-a3e4-48571e98d4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:39.450351 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222e4ab-c38a-4610-8dfe-7cd281086cc3 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:39.466778 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-c14ffb42-1d0d-4c10-a3e4-48571e98d4a5 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:39.468441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c14ffb42-1d0d-4c10-a3e4-48571e98d4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:39.479653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:39.495151 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786fc1f-ea54-4f22-a34f-5856a6f9f6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:39.496293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786fc1f-ea54-4f22-a34f-5856a6f9f6dc None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:39.496798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786fc1f-ea54-4f22-a34f-5856a6f9f6dc None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:39.497511 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786fc1f-ea54-4f22-a34f-5856a6f9f6dc None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:39.498777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4786fc1f-ea54-4f22-a34f-5856a6f9f6dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:39.499512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4786fc1f-ea54-4f22-a34f-5856a6f9f6dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:39.501685 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4786fc1f-ea54-4f22-a34f-5856a6f9f6dc None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:39.519707 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 368/761] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:39.520993 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786fc1f-ea54-4f22-a34f-5856a6f9f6dc None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:39.550793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 394/762] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:56:39.551931 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ffb42-1d0d-4c10-a3e4-48571e98d4a5 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:39.665781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ffb42-1d0d-4c10-a3e4-48571e98d4a5 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:39.689599 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786fc1f-ea54-4f22-a34f-5856a6f9f6dc None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:39.701964 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:39.745839 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d07a4-0d37-484a-b032-6b46698d7344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:39.746833 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d07a4-0d37-484a-b032-6b46698d7344 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:39.747336 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d07a4-0d37-484a-b032-6b46698d7344 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:39.747794 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d07a4-0d37-484a-b032-6b46698d7344 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:39.751256 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111e5fa-071f-41f3-8730-28ad0c404f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:39.751953 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111e5fa-071f-41f3-8730-28ad0c404f1b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:39.752296 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111e5fa-071f-41f3-8730-28ad0c404f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:39.753147 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111e5fa-071f-41f3-8730-28ad0c404f1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:39.762514 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111e5fa-071f-41f3-8730-28ad0c404f1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:39.762514 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8111e5fa-071f-41f3-8730-28ad0c404f1b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:39.763708 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d07a4-0d37-484a-b032-6b46698d7344 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:39.765784 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41d07a4-0d37-484a-b032-6b46698d7344 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:39.773951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-8111e5fa-071f-41f3-8730-28ad0c404f1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:39.773951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:39.773951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:39.773951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:39.773951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:39.773951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:39.773951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:39.773951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:39.773951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:39.773951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:39.773951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:39.773951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:39.773951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:39.773951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:39.773951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:39.774849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:39.774849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:39.774849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:39.774849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:39.774849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:39.774849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:39.774849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:39.774849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:39.774849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:39.774849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:39.774849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:39.774849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:39.774849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:39.774849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:39.774849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:39.774849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:39.774849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:39.775779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:39.775779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:39.775779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:39.775779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:39.775779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:39.775779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:39.775779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:39.775779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:39.775779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 369/763] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:39.785675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-a41d07a4-0d37-484a-b032-6b46698d7344 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:39.785675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:39.785675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:39.785675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:39.785675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:39.785675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:39.785675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:39.785675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:39.785675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:39.785675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:39.785675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:39.785675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:39.785675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:39.785675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:39.785675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:39.786577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:39.786577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:39.786577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:39.786577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:39.786577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:39.786577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:39.786577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:39.786577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:39.786577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:39.786577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:39.786577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:39.786577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:39.786577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:39.786577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:39.786577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:39.786577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:39.786577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:39.787501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:39.787501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:39.787501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:39.787501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:39.787501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:39.787501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:39.787501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:39.787501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:39.788354 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111e5fa-071f-41f3-8730-28ad0c404f1b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:39.789940 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 395/764] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:39.795988 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d07a4-0d37-484a-b032-6b46698d7344 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:39.835642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:39.889429 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111e5fa-071f-41f3-8730-28ad0c404f1b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:39.924277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab383397-4949-4116-8716-bb3651d75976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:39.925126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab383397-4949-4116-8716-bb3651d75976 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:39.925126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab383397-4949-4116-8716-bb3651d75976 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:39.925412 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab383397-4949-4116-8716-bb3651d75976 None admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:39.926100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab383397-4949-4116-8716-bb3651d75976 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:39.926401 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab383397-4949-4116-8716-bb3651d75976 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:39.927970 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab383397-4949-4116-8716-bb3651d75976 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:39.936928 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d07a4-0d37-484a-b032-6b46698d7344 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:39.942556 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 370/765] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:39.943506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab383397-4949-4116-8716-bb3651d75976 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:39.980962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c1dcb-0ba4-43aa-8743-c1e296023049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:39.981654 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c1dcb-0ba4-43aa-8743-c1e296023049 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:39.981935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c1dcb-0ba4-43aa-8743-c1e296023049 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:39.982497 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c1dcb-0ba4-43aa-8743-c1e296023049 None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:39.983474 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c1dcb-0ba4-43aa-8743-c1e296023049 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:39.983782 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c1dcb-0ba4-43aa-8743-c1e296023049 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:39.985213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c1dcb-0ba4-43aa-8743-c1e296023049 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:40.001571 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 396/766] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:40.003033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c1dcb-0ba4-43aa-8743-c1e296023049 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:40.047443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:40.070909 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab383397-4949-4116-8716-bb3651d75976 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:40.074177 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:40.111856 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c1dcb-0ba4-43aa-8743-c1e296023049 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:40.117091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b7e30-1ad5-4910-9bdf-cdf6b034dc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:40.117737 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b7e30-1ad5-4910-9bdf-cdf6b034dc8a None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:40.118020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b7e30-1ad5-4910-9bdf-cdf6b034dc8a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:40.118295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5b7e30-1ad5-4910-9bdf-cdf6b034dc8a None admin] PATH_INFO: `/v3/services/network` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:40.118882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b7e30-1ad5-4910-9bdf-cdf6b034dc8a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:40.120367 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5b7e30-1ad5-4910-9bdf-cdf6b034dc8a None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:40.132777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-1b5b7e30-1ad5-4910-9bdf-cdf6b034dc8a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 08:56:40.132777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:40.132777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:56:40.132777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:56:40.132777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:56:40.132777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:56:40.132777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:56:40.132777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:40.132777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 08:56:40.132777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:40.132777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:56:40.132777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:40.132777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:40.132777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:40.132777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:40.132777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:40.132777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:40.133775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:40.133775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:40.133775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:40.133775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:40.133775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:40.133775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:40.133775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:56:40.133775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:56:40.133775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:56:40.133775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:56:40.133775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:40.133775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:40.133775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:40.133775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:40.133775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:40.133775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:40.134630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:40.134630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:40.134630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:40.134630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:40.134630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:40.134630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:40.134630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:40.134630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:40.134630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:40.134630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:40.134630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:56:40.134630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:40.134630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 08:56:40.134630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:40.139152 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 371/767] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:39 2022] GET /identity/v3/services/network => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:40.141119 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b7e30-1ad5-4910-9bdf-cdf6b034dc8a None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:40.160876 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a75c04-08ef-4e99-987a-cf71c23b53db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:40.162107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a75c04-08ef-4e99-987a-cf71c23b53db None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:40.162983 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a75c04-08ef-4e99-987a-cf71c23b53db None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:40.163509 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a75c04-08ef-4e99-987a-cf71c23b53db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:40.164459 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a75c04-08ef-4e99-987a-cf71c23b53db None admin] RBAC: Adding query filter params (role.id=637ed3cd3d484839ab21de02c2bbe293, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=76ebecce283f491e82a961d7f739878f) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:40.164972 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a75c04-08ef-4e99-987a-cf71c23b53db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:40.167423 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a75c04-08ef-4e99-987a-cf71c23b53db None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:40.176373 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 397/768] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:56:40 2022] GET /identity/v3/role_assignments?user.id=76ebecce283f491e82a961d7f739878f&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=637ed3cd3d484839ab21de02c2bbe293 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:40.178101 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a75c04-08ef-4e99-987a-cf71c23b53db None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:40.304306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a75c04-08ef-4e99-987a-cf71c23b53db None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:40.307469 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:40.362644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5b7e30-1ad5-4910-9bdf-cdf6b034dc8a None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:40.371912 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7e352-3989-4943-b6d4-fe24c6e64fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:40.372551 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7e352-3989-4943-b6d4-fe24c6e64fcb None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:40.372835 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7e352-3989-4943-b6d4-fe24c6e64fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:40.373107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7e352-3989-4943-b6d4-fe24c6e64fcb None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:40.373747 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7e352-3989-4943-b6d4-fe24c6e64fcb None admin] RBAC: Adding query filter params (name=network) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:40.374049 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7e352-3989-4943-b6d4-fe24c6e64fcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:40.375519 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7e352-3989-4943-b6d4-fe24c6e64fcb None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:40.397105 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 398/769] 10.222.0.36 () {56 vars in 1112 bytes} [Fri Sep 2 08:56:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:40.404460 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7e352-3989-4943-b6d4-fe24c6e64fcb None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:40.408189 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fab8fc-b751-4697-8905-136a4a80fdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:40.409115 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fab8fc-b751-4697-8905-136a4a80fdfc None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:40.409801 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fab8fc-b751-4697-8905-136a4a80fdfc None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:40.410277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fab8fc-b751-4697-8905-136a4a80fdfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:40.411237 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fab8fc-b751-4697-8905-136a4a80fdfc None admin] RBAC: Adding query filter params (role.id=a0a76de72fba4036bc1e3e64df717c7f, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=92318948dcf1407b9fc8aa5c199787ae) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:40.411769 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fab8fc-b751-4697-8905-136a4a80fdfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:40.413394 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fab8fc-b751-4697-8905-136a4a80fdfc None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:40.420863 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 372/770] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:56:40 2022] GET /identity/v3/role_assignments?user.id=92318948dcf1407b9fc8aa5c199787ae&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=a0a76de72fba4036bc1e3e64df717c7f => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:40.588444 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7e352-3989-4943-b6d4-fe24c6e64fcb None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:40.641173 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec9125-2834-48b3-bde4-9e16f38ae7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:40.641788 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec9125-2834-48b3-bde4-9e16f38ae7da None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:40.642067 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec9125-2834-48b3-bde4-9e16f38ae7da None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:40.642333 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec9125-2834-48b3-bde4-9e16f38ae7da None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:40.642965 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec9125-2834-48b3-bde4-9e16f38ae7da None admin] RBAC: Adding query filter params (type=network) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:40.643327 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec9125-2834-48b3-bde4-9e16f38ae7da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:40.644694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec9125-2834-48b3-bde4-9e16f38ae7da None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:40.655478 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 399/771] 10.222.0.36 () {56 vars in 1112 bytes} [Fri Sep 2 08:56:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:40.661901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fab8fc-b751-4697-8905-136a4a80fdfc None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:40.707995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:40.745994 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dddb0b1-d637-4ce4-a13f-ccb41c8c95f3 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:40.746328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dddb0b1-d637-4ce4-a13f-ccb41c8c95f3 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:40.746548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dddb0b1-d637-4ce4-a13f-ccb41c8c95f3 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:40.748558 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 400/772] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:56:40.754743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fcb96-ac94-48a8-85db-017978afe39a None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:40.755038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fcb96-ac94-48a8-85db-017978afe39a None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:40.755346 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21fcb96-ac94-48a8-85db-017978afe39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:40.777420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-a21fcb96-ac94-48a8-85db-017978afe39a None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:40.778034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a21fcb96-ac94-48a8-85db-017978afe39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:40.821627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:40.830007 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fab8fc-b751-4697-8905-136a4a80fdfc None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:40.887133 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 401/773] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 08:56:40.888553 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc903f4d-7958-4421-a9f1-ab21ee4cf218 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:40.888834 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc903f4d-7958-4421-a9f1-ab21ee4cf218 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:40.889099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc903f4d-7958-4421-a9f1-ab21ee4cf218 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:40.892187 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc45e67-ee87-4cc1-867e-b9fe2f4176d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:40.893050 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc45e67-ee87-4cc1-867e-b9fe2f4176d3 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:40.894139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc45e67-ee87-4cc1-867e-b9fe2f4176d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:40.894616 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc45e67-ee87-4cc1-867e-b9fe2f4176d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:40.894851 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 402/774] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:40.896062 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc45e67-ee87-4cc1-867e-b9fe2f4176d3 None admin] RBAC: Adding query filter params (interface=public, service_id=402680e3f1954b188d1682ac78d6b6fb, region_id=RegionOne) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:40.896423 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c2f15-2625-445a-954d-0fe023d94ba2 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:40.896705 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c2f15-2625-445a-954d-0fe023d94ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:40.896976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c2f15-2625-445a-954d-0fe023d94ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:40.897218 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc45e67-ee87-4cc1-867e-b9fe2f4176d3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:40.898840 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc45e67-ee87-4cc1-867e-b9fe2f4176d3 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:40.908034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 373/775] 10.222.0.36 () {56 vars in 1251 bytes} [Fri Sep 2 08:56:40 2022] GET /identity/v3/endpoints?service_id=402680e3f1954b188d1682ac78d6b6fb&interface=public®ion_id=RegionOne => generated 194 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:40.914951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc45e67-ee87-4cc1-867e-b9fe2f4176d3 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:40.926188 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-871c2f15-2625-445a-954d-0fe023d94ba2 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:40.927404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-871c2f15-2625-445a-954d-0fe023d94ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:41.052625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 403/776] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 08:56:41.063222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:41.078739 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc45e67-ee87-4cc1-867e-b9fe2f4176d3 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:41.145364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8cfb8-f7b6-41b1-a7d5-98cdfb0683ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:41.145902 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8cfb8-f7b6-41b1-a7d5-98cdfb0683ae None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:41.146184 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8cfb8-f7b6-41b1-a7d5-98cdfb0683ae None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:41.146460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8cfb8-f7b6-41b1-a7d5-98cdfb0683ae None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:41.147090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8cfb8-f7b6-41b1-a7d5-98cdfb0683ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:41.148451 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8cfb8-f7b6-41b1-a7d5-98cdfb0683ae None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:41.162101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 374/777] 10.222.0.36 () {56 vars in 1087 bytes} [Fri Sep 2 08:56:40 2022] GET /identity/v3/services => generated 772 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:41.174490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c2f15-2625-445a-954d-0fe023d94ba2 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:41.303444 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:41.317702 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c2f15-2625-445a-954d-0fe023d94ba2 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:41.347501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8cfb8-f7b6-41b1-a7d5-98cdfb0683ae None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:41.359163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba112125-c624-45d5-a202-8ccc9121cd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:41.359811 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba112125-c624-45d5-a202-8ccc9121cd6a None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:41.360089 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba112125-c624-45d5-a202-8ccc9121cd6a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:41.360359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba112125-c624-45d5-a202-8ccc9121cd6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:41.372333 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba112125-c624-45d5-a202-8ccc9121cd6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:41.377120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-ba112125-c624-45d5-a202-8ccc9121cd6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:56:41.377120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:41.377120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:41.377120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:41.377120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:41.377120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:41.377120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:41.377120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:41.377120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:41.377120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:41.377120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:41.377120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:41.377120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:41.377120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:41.377120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:41.377120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:41.378566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:41.378566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:41.378566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:41.378566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:41.378566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:41.378566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:41.378566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:41.378566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:41.378566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:41.378566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:41.378566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:41.378566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:41.378566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:41.378566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:41.378566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:41.378566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:41.379550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:41.379550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:41.379550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:41.379550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:41.379550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:41.379550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:41.379550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:41.379550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:41.379550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:41.379550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:41.379550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:56:41.379550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:41.379550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 404/778] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:56:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:41.384312 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba112125-c624-45d5-a202-8ccc9121cd6a None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:41.479271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8cfb8-f7b6-41b1-a7d5-98cdfb0683ae None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:41.519408 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fb5eb-5330-4a27-9853-b60defebdb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:41.520591 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fb5eb-5330-4a27-9853-b60defebdb62 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:41.521833 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fb5eb-5330-4a27-9853-b60defebdb62 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:41.522874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fb5eb-5330-4a27-9853-b60defebdb62 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:41.523841 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03fb5eb-5330-4a27-9853-b60defebdb62 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:41.527887 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03fb5eb-5330-4a27-9853-b60defebdb62 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:41.546395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-d03fb5eb-5330-4a27-9853-b60defebdb62 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 08:56:41.546395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:41.546395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:56:41.546395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:56:41.546395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:56:41.546395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:56:41.546395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:56:41.546395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:41.546395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 08:56:41.546395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:41.546395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:56:41.546395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:41.546395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:41.546395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:41.546395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:41.546395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:41.546395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:41.547498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:41.547498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:41.547498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:41.547498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:41.547498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:41.547498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:41.547498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:56:41.547498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:56:41.547498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:56:41.547498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:56:41.547498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:41.547498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:41.547498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:41.547498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:41.547498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:41.547498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:41.548335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:41.548335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:41.548335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:41.548335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:41.548335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:41.548335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:41.548335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:41.548335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:41.548335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:41.548335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:41.548335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:56:41.548335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:41.548335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 08:56:41.548335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:41.551953 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 375/779] 10.222.0.36 () {56 vars in 1129 bytes} [Fri Sep 2 08:56:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:41.560452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fb5eb-5330-4a27-9853-b60defebdb62 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:41.582308 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba112125-c624-45d5-a202-8ccc9121cd6a None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:41.609692 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:41.634782 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21102051-7ade-43bb-9dd5-1547e6878fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:41.635413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21102051-7ade-43bb-9dd5-1547e6878fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:41.635685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21102051-7ade-43bb-9dd5-1547e6878fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:41.635950 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21102051-7ade-43bb-9dd5-1547e6878fa0 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:41.636608 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21102051-7ade-43bb-9dd5-1547e6878fa0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:41.636903 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21102051-7ade-43bb-9dd5-1547e6878fa0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:41.638270 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21102051-7ade-43bb-9dd5-1547e6878fa0 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:41.654763 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 405/780] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:56:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:41.659605 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21102051-7ade-43bb-9dd5-1547e6878fa0 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:41.677904 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:41.752045 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fb5eb-5330-4a27-9853-b60defebdb62 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:41.791613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9efdd-2653-45c5-ade9-af465a4b74b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:41.792501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9efdd-2653-45c5-ade9-af465a4b74b7 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:41.792787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9efdd-2653-45c5-ade9-af465a4b74b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:41.793060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9efdd-2653-45c5-ade9-af465a4b74b7 None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:41.793851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9efdd-2653-45c5-ade9-af465a4b74b7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:41.794165 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9efdd-2653-45c5-ade9-af465a4b74b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:41.796541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9efdd-2653-45c5-ade9-af465a4b74b7 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:41.802312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 376/781] 10.222.0.36 () {56 vars in 1124 bytes} [Fri Sep 2 08:56:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:41.810821 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9efdd-2653-45c5-ade9-af465a4b74b7 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:41.847787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21102051-7ade-43bb-9dd5-1547e6878fa0 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:41.909212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234eb01-7910-44ab-902f-9f2fd65da46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:41.909968 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234eb01-7910-44ab-902f-9f2fd65da46a None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:41.910249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234eb01-7910-44ab-902f-9f2fd65da46a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:41.910520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234eb01-7910-44ab-902f-9f2fd65da46a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:41.928328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234eb01-7910-44ab-902f-9f2fd65da46a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:41.931478 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234eb01-7910-44ab-902f-9f2fd65da46a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:41.958764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-d234eb01-7910-44ab-902f-9f2fd65da46a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:41.958764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:41.958764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:41.958764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:41.958764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:41.958764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:41.958764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:41.958764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:41.958764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:41.958764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:41.958764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:41.958764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:41.958764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:41.958764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:41.958764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:41.958764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:41.961044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:41.961044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:41.961044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:41.961044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:41.961044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:41.961044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:41.961044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:41.961044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:41.961044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:41.961044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:41.961044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:41.961044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:41.961044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:41.961044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:41.961044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:41.961044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:41.961044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:41.963216 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:41.963216 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:41.963216 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:41.963613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:41.967508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 406/782] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:41.973536 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234eb01-7910-44ab-902f-9f2fd65da46a None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:41.975730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9efdd-2653-45c5-ade9-af465a4b74b7 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:42.047803 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265253f-a2a9-4007-bc49-94d32b9fe50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:42.048442 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265253f-a2a9-4007-bc49-94d32b9fe50c None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:42.048719 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265253f-a2a9-4007-bc49-94d32b9fe50c None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:42.048984 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a265253f-a2a9-4007-bc49-94d32b9fe50c None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:42.050235 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265253f-a2a9-4007-bc49-94d32b9fe50c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:42.050536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265253f-a2a9-4007-bc49-94d32b9fe50c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:42.051919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a265253f-a2a9-4007-bc49-94d32b9fe50c None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:42.066897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 377/783] 10.222.0.36 () {56 vars in 1124 bytes} [Fri Sep 2 08:56:41 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:42.068152 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265253f-a2a9-4007-bc49-94d32b9fe50c None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:42.152828 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:42.193907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a265253f-a2a9-4007-bc49-94d32b9fe50c None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:42.231339 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234eb01-7910-44ab-902f-9f2fd65da46a None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:42.254520 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847adca-baad-41c0-8bf5-89f3dcf8e5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:42.255226 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847adca-baad-41c0-8bf5-89f3dcf8e5e2 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:42.255511 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847adca-baad-41c0-8bf5-89f3dcf8e5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:42.255778 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847adca-baad-41c0-8bf5-89f3dcf8e5e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:42.256484 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847adca-baad-41c0-8bf5-89f3dcf8e5e2 None admin] RBAC: Adding query filter params (interface=public, service_id=b122bd1c298849c492e20de800848ff6, region_id=RegionOne) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:42.256787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847adca-baad-41c0-8bf5-89f3dcf8e5e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:42.258154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847adca-baad-41c0-8bf5-89f3dcf8e5e2 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:42.270207 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 378/784] 10.222.0.36 () {56 vars in 1251 bytes} [Fri Sep 2 08:56:42 2022] GET /identity/v3/endpoints?service_id=b122bd1c298849c492e20de800848ff6&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:42.288143 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847adca-baad-41c0-8bf5-89f3dcf8e5e2 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:42.307214 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8611f40-f286-497c-929d-2cef612a0853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:42.307815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8611f40-f286-497c-929d-2cef612a0853 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:42.308092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8611f40-f286-497c-929d-2cef612a0853 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:42.308361 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8611f40-f286-497c-929d-2cef612a0853 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:42.309257 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8611f40-f286-497c-929d-2cef612a0853 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:42.309555 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8611f40-f286-497c-929d-2cef612a0853 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:42.313226 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8611f40-f286-497c-929d-2cef612a0853 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:42.331106 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 407/785] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:42.331404 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:42.343104 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8611f40-f286-497c-929d-2cef612a0853 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:42.447849 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8611f40-f286-497c-929d-2cef612a0853 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:42.493824 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:42.506794 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847adca-baad-41c0-8bf5-89f3dcf8e5e2 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:42.506986 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4500145-3167-4578-952e-ff31e24ebe25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:42.507677 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4500145-3167-4578-952e-ff31e24ebe25 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:42.507959 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4500145-3167-4578-952e-ff31e24ebe25 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:42.508236 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4500145-3167-4578-952e-ff31e24ebe25 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:42.515737 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4500145-3167-4578-952e-ff31e24ebe25 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:42.517188 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4500145-3167-4578-952e-ff31e24ebe25 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:42.530030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-e4500145-3167-4578-952e-ff31e24ebe25 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 08:56:42.530030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:42.530030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:42.530030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:42.530030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:42.530030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:42.530030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:42.530030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:42.530030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:42.530030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:42.530030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:42.530030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:42.530030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:42.530030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:42.530030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:42.530030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:42.531102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:42.531102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:42.531102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:42.531102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:42.531102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:42.531102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:42.531102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:42.531102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:42.531102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:42.531102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:42.531102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:42.531102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:42.531102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:42.531102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:42.531102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:42.531102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:42.531102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:42.531102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:42.532077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:42.532077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:42.532077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:42.532077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:42.532077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:42.532077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:42.532077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:42.532077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:42.532077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 08:56:42.532077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:42.536700 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 408/786] 10.222.0.36 () {56 vars in 1093 bytes} [Fri Sep 2 08:56:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:42.538189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b161c0-8162-4b42-a7ce-7234f356ac4f None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:42.538475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b161c0-8162-4b42-a7ce-7234f356ac4f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:42.538763 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b161c0-8162-4b42-a7ce-7234f356ac4f None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:42.540412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 409/787] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:42.541145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b161c0-8162-4b42-a7ce-7234f356ac4f None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:42.552606 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2577ab-ec67-4015-9d2c-0f887342bc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:42.553662 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2577ab-ec67-4015-9d2c-0f887342bc05 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:42.554150 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2577ab-ec67-4015-9d2c-0f887342bc05 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:42.554702 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2577ab-ec67-4015-9d2c-0f887342bc05 None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:42.555570 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2577ab-ec67-4015-9d2c-0f887342bc05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:42.557333 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2577ab-ec67-4015-9d2c-0f887342bc05 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:42.565828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 379/788] 10.222.0.36 () {56 vars in 1087 bytes} [Fri Sep 2 08:56:42 2022] GET /identity/v3/services => generated 772 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:42.567233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199dda52-4572-44e7-a431-c7f395b9617a None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:42.567720 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199dda52-4572-44e7-a431-c7f395b9617a None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:42.568287 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199dda52-4572-44e7-a431-c7f395b9617a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:42.590464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-199dda52-4572-44e7-a431-c7f395b9617a None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:42.592190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-199dda52-4572-44e7-a431-c7f395b9617a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:42.681005 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 380/789] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:56:42.683553 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768d7de-1dff-4b4d-b1ac-e51cd5dd6d07 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:42.684333 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768d7de-1dff-4b4d-b1ac-e51cd5dd6d07 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:42.684797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768d7de-1dff-4b4d-b1ac-e51cd5dd6d07 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:42.687206 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 381/790] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:42.699101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee926c6e-b5cb-429e-9c62-941e157a28f5 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:42.699101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee926c6e-b5cb-429e-9c62-941e157a28f5 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:42.699101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee926c6e-b5cb-429e-9c62-941e157a28f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:42.701771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b161c0-8162-4b42-a7ce-7234f356ac4f None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:42.737945 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-ee926c6e-b5cb-429e-9c62-941e157a28f5 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:42.741149 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ee926c6e-b5cb-429e-9c62-941e157a28f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:42.749594 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:42.768010 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a7d9a-f834-4f38-b665-8837180cb0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:42.768654 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a7d9a-f834-4f38-b665-8837180cb0ec None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:42.768929 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a7d9a-f834-4f38-b665-8837180cb0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:42.769196 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a7d9a-f834-4f38-b665-8837180cb0ec None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:42.769919 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a7d9a-f834-4f38-b665-8837180cb0ec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:42.770217 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a7d9a-f834-4f38-b665-8837180cb0ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:42.771694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873a7d9a-f834-4f38-b665-8837180cb0ec None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:42.795713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 410/791] 10.222.0.36 () {56 vars in 1133 bytes} [Fri Sep 2 08:56:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:42.801430 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a7d9a-f834-4f38-b665-8837180cb0ec None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:42.844165 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 382/792] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 08:56:42.857539 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:42.931369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873a7d9a-f834-4f38-b665-8837180cb0ec None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:42.961617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee926c6e-b5cb-429e-9c62-941e157a28f5 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:42.974659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa01d7-1d34-47c2-b375-52278bf2f60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:42.975610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa01d7-1d34-47c2-b375-52278bf2f60b None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:42.976111 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa01d7-1d34-47c2-b375-52278bf2f60b None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:42.976581 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa01d7-1d34-47c2-b375-52278bf2f60b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:43.104916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa01d7-1d34-47c2-b375-52278bf2f60b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:43.106728 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aa01d7-1d34-47c2-b375-52278bf2f60b None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:43.120568 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee926c6e-b5cb-429e-9c62-941e157a28f5 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:43.160664 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384a61a-706a-496c-9fa3-30a2d8f8561a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:43.161296 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384a61a-706a-496c-9fa3-30a2d8f8561a None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:43.161572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384a61a-706a-496c-9fa3-30a2d8f8561a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:43.161841 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384a61a-706a-496c-9fa3-30a2d8f8561a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:43.165542 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:43.171303 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:43.175834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f384a61a-706a-496c-9fa3-30a2d8f8561a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:43.182517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-f384a61a-706a-496c-9fa3-30a2d8f8561a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:43.182517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:43.182517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:43.182517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:43.182517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:43.182517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:43.182517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:43.182517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:43.182517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:43.182517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:43.182517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:43.182517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:43.182517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:43.182517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:43.182517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:43.182517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:43.183519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:43.183519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:43.183519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:43.183519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:43.183519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:43.183519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:43.183519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:43.183519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:43.183519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:43.183519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:43.183519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:43.183519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:43.183519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:43.183519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:43.183519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:43.183519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:43.184613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:43.184613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:43.184613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:43.184613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:43.184613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:43.184613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:43.184613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:43.184613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:43.184613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:43.184613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:43.184613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:43.184613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:43.187457 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 383/793] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:56:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:43.188641 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384a61a-706a-496c-9fa3-30a2d8f8561a None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:43.234379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 411/794] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:42 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 02 08:56:43.235512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa01d7-1d34-47c2-b375-52278bf2f60b None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:43.318379 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384a61a-706a-496c-9fa3-30a2d8f8561a None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:43.347864 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aa01d7-1d34-47c2-b375-52278bf2f60b None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:43.359315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55156f94-42b6-497b-8306-f92c7750955c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:43.360260 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55156f94-42b6-497b-8306-f92c7750955c None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:43.361258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55156f94-42b6-497b-8306-f92c7750955c None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:43.361739 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55156f94-42b6-497b-8306-f92c7750955c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:43.373721 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55156f94-42b6-497b-8306-f92c7750955c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:43.379304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-55156f94-42b6-497b-8306-f92c7750955c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:56:43.379304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:43.379304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:43.379304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:43.379304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:43.379304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:43.379304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:43.379304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:43.379304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:43.379304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:43.379304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:43.379304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:43.379304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:43.379304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:43.379304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:43.379304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:43.380253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:43.380253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:43.380253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:43.380253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:43.380253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:43.380253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:43.380253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:43.380253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:43.380253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:43.380253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:43.380253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:43.380253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:43.380253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:43.380253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:43.380253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:43.380253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:43.381101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:43.381101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:43.381101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:43.381101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:43.381101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:43.381101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:43.381101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:43.381101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:43.381101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:43.381101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:43.381101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:56:43.381101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:43.383734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 384/795] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:56:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:43.384718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55156f94-42b6-497b-8306-f92c7750955c None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:43.390319 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033bb70-28f6-43e5-be9e-bdf3ddd049fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:43.391046 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033bb70-28f6-43e5-be9e-bdf3ddd049fa None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:43.391458 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033bb70-28f6-43e5-be9e-bdf3ddd049fa None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:43.391843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033bb70-28f6-43e5-be9e-bdf3ddd049fa None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:43.392613 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033bb70-28f6-43e5-be9e-bdf3ddd049fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:43.393527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033bb70-28f6-43e5-be9e-bdf3ddd049fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:43.395097 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033bb70-28f6-43e5-be9e-bdf3ddd049fa None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:43.404239 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 412/796] 10.222.0.36 () {56 vars in 1103 bytes} [Fri Sep 2 08:56:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:43.405212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033bb70-28f6-43e5-be9e-bdf3ddd049fa None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:43.408053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:43.454453 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:43.500811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55156f94-42b6-497b-8306-f92c7750955c None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:43.512068 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033bb70-28f6-43e5-be9e-bdf3ddd049fa None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:43.537084 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9702b20-f7d8-4cc3-9f58-e5e19a8af850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:43.537797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9702b20-f7d8-4cc3-9f58-e5e19a8af850 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:43.538139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9702b20-f7d8-4cc3-9f58-e5e19a8af850 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:43.538477 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9702b20-f7d8-4cc3-9f58-e5e19a8af850 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:43.547022 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07fcf7-d48d-477a-8763-8580a0158289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:43.547717 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07fcf7-d48d-477a-8763-8580a0158289 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:43.548071 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07fcf7-d48d-477a-8763-8580a0158289 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:43.548347 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff07fcf7-d48d-477a-8763-8580a0158289 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:43.549001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07fcf7-d48d-477a-8763-8580a0158289 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:43.549103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9702b20-f7d8-4cc3-9f58-e5e19a8af850 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:43.549304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07fcf7-d48d-477a-8763-8580a0158289 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:43.550660 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff07fcf7-d48d-477a-8763-8580a0158289 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:43.550763 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9702b20-f7d8-4cc3-9f58-e5e19a8af850 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:43.559575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 413/797] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:56:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:43.560368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07fcf7-d48d-477a-8763-8580a0158289 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:43.570600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-d9702b20-f7d8-4cc3-9f58-e5e19a8af850 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:43.570600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:43.570600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:43.570600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:43.570600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:43.570600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:43.570600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:43.570600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:43.570600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:43.570600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:43.570600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:43.570600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:43.570600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:43.570600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:43.570600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:43.570600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:43.571771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:43.571771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:43.571771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:43.571771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:43.571771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:43.571771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:43.571771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:43.571771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:43.571771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:43.571771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:43.571771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:43.571771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:43.571771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:43.571771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:43.571771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:43.571771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:43.571771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:43.572860 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:43.572860 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:43.572860 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:43.575215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 385/798] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:43.576450 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f32dac-db3a-475f-9350-9c6d47cea3a0 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:43.576733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f32dac-db3a-475f-9350-9c6d47cea3a0 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:43.577008 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f32dac-db3a-475f-9350-9c6d47cea3a0 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:43.578259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 386/799] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:43.578947 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f32dac-db3a-475f-9350-9c6d47cea3a0 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:43.634422 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:43.674059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff07fcf7-d48d-477a-8763-8580a0158289 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:43.695049 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f32dac-db3a-475f-9350-9c6d47cea3a0 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:43.711369 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:43.736250 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6de5c3-633f-443a-b920-93e6ef787e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:43.737623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6de5c3-633f-443a-b920-93e6ef787e4a None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:43.738010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6de5c3-633f-443a-b920-93e6ef787e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:43.738381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6de5c3-633f-443a-b920-93e6ef787e4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:43.750697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6de5c3-633f-443a-b920-93e6ef787e4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:43.752429 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6de5c3-633f-443a-b920-93e6ef787e4a None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:43.768234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-9b6de5c3-633f-443a-b920-93e6ef787e4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:43.768234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:43.768234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:43.768234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:43.768234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:43.768234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:43.768234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:43.768234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:43.768234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:43.768234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:43.768234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:43.768234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:43.768234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:43.768234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:43.768234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:43.768234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:43.769435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:43.769435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:43.769435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:43.769435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:43.769435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:43.769435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:43.769435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:43.769435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:43.769435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:43.769435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:43.769435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:43.769435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:43.769435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:43.769435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:43.769435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:43.769435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:43.769435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:43.770527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:43.770527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:43.770527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:43.776544 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 387/800] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:43.776544 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6de5c3-633f-443a-b920-93e6ef787e4a None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:43.824999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eed86fe-a86c-476d-bfcd-d8a6b0b032d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:43.825631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed86fe-a86c-476d-bfcd-d8a6b0b032d1 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:43.825913 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed86fe-a86c-476d-bfcd-d8a6b0b032d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:43.826193 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eed86fe-a86c-476d-bfcd-d8a6b0b032d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:43.836952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eed86fe-a86c-476d-bfcd-d8a6b0b032d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:43.838413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eed86fe-a86c-476d-bfcd-d8a6b0b032d1 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:43.853610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-0eed86fe-a86c-476d-bfcd-d8a6b0b032d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:43.853610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:43.853610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:43.853610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:43.853610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:43.853610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:43.853610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:43.853610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:43.853610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:43.853610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:43.853610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:43.853610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:43.853610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:43.853610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:43.853610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:43.853610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:43.854603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:43.854603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:43.854603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:43.854603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:43.854603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:43.854603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:43.854603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:43.854603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:43.854603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:43.854603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:43.854603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:43.854603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:43.854603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:43.854603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:43.854603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:43.854603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:43.854603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:43.856075 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:43.856075 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:43.856075 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:43.859418 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 414/801] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:43.860670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3265db5-c224-4982-b853-073c15f546eb None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:43.860959 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3265db5-c224-4982-b853-073c15f546eb None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:43.861228 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3265db5-c224-4982-b853-073c15f546eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:43.879048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-b3265db5-c224-4982-b853-073c15f546eb None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:43.880270 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-b3265db5-c224-4982-b853-073c15f546eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:43.881558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6de5c3-633f-443a-b920-93e6ef787e4a None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:43.919141 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02a3e1-c8be-46b5-ac99-e962937ac278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:43.919718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02a3e1-c8be-46b5-ac99-e962937ac278 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:43.920002 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02a3e1-c8be-46b5-ac99-e962937ac278 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:43.920275 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02a3e1-c8be-46b5-ac99-e962937ac278 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:43.920922 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02a3e1-c8be-46b5-ac99-e962937ac278 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:43.921231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02a3e1-c8be-46b5-ac99-e962937ac278 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:43.922626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02a3e1-c8be-46b5-ac99-e962937ac278 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:43.940858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 388/802] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:43.940858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02a3e1-c8be-46b5-ac99-e962937ac278 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:43.954433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 415/803] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:56:43.956075 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3265db5-c224-4982-b853-073c15f546eb None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:43.976210 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:44.020641 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:44.064716 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02a3e1-c8be-46b5-ac99-e962937ac278 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:44.085803 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3265db5-c224-4982-b853-073c15f546eb None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:44.100449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca120588-e2cd-4d09-aa40-d5ec6c6e74f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:44.101261 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca120588-e2cd-4d09-aa40-d5ec6c6e74f2 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:44.101420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca120588-e2cd-4d09-aa40-d5ec6c6e74f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:44.101829 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca120588-e2cd-4d09-aa40-d5ec6c6e74f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:44.102570 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca120588-e2cd-4d09-aa40-d5ec6c6e74f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:44.102939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca120588-e2cd-4d09-aa40-d5ec6c6e74f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:44.104361 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca120588-e2cd-4d09-aa40-d5ec6c6e74f2 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:44.119500 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 389/804] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:44.121053 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca120588-e2cd-4d09-aa40-d5ec6c6e74f2 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:44.127402 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7812e-560d-4eca-b478-569850657471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:44.128259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7812e-560d-4eca-b478-569850657471 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:44.128744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7812e-560d-4eca-b478-569850657471 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:44.129232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7812e-560d-4eca-b478-569850657471 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:44.130073 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7812e-560d-4eca-b478-569850657471 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:44.130594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7812e-560d-4eca-b478-569850657471 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:44.132546 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7812e-560d-4eca-b478-569850657471 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:44.146008 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 416/805] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:44.146904 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7812e-560d-4eca-b478-569850657471 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:44.211047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:44.227823 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca120588-e2cd-4d09-aa40-d5ec6c6e74f2 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:44.258186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7812e-560d-4eca-b478-569850657471 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:44.262536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33687937-8654-4ab0-b0e5-c35fe64eab89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:44.263232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33687937-8654-4ab0-b0e5-c35fe64eab89 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:44.263516 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33687937-8654-4ab0-b0e5-c35fe64eab89 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:44.263793 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33687937-8654-4ab0-b0e5-c35fe64eab89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:44.265912 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:44.272359 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33687937-8654-4ab0-b0e5-c35fe64eab89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:44.274070 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33687937-8654-4ab0-b0e5-c35fe64eab89 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:44.286731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-33687937-8654-4ab0-b0e5-c35fe64eab89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:44.286731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:44.286731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:44.286731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:44.286731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:44.286731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:44.286731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:44.286731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:44.286731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:44.286731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:44.286731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:44.286731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:44.286731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:44.286731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:44.286731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:44.287681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:44.287681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:44.287681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:44.287681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:44.287681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:44.287681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:44.287681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:44.287681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:44.287681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:44.287681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:44.287681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:44.287681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:44.287681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:44.287681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:44.287681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:44.287681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:44.287681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:44.288584 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:44.288584 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:44.288584 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:44.288584 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:44.288584 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:44.288584 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:44.288584 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:44.288584 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:44.288584 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 390/806] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:44.288584 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33687937-8654-4ab0-b0e5-c35fe64eab89 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:44.300399 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbdfaf-1821-40de-8a8f-3a77655e025b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:44.301249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbdfaf-1821-40de-8a8f-3a77655e025b None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:44.301357 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbdfaf-1821-40de-8a8f-3a77655e025b None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:44.301579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbdfaf-1821-40de-8a8f-3a77655e025b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:44.309760 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbdfaf-1821-40de-8a8f-3a77655e025b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:44.311678 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbdfaf-1821-40de-8a8f-3a77655e025b None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:44.324234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-0fbbdfaf-1821-40de-8a8f-3a77655e025b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 02 08:56:44.324234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:44.324234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:44.324234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:44.324234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:44.324234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:44.324234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:44.324234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:44.324234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:44.324234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:44.324234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:44.324234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:44.324234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:44.324234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:44.324234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:44.324234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:44.325207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:44.325207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:44.325207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:44.325207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:44.325207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:44.325207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:44.325207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:44.325207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:44.325207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:44.325207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:44.325207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:44.325207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:44.325207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:44.325207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:44.325207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:44.325207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:44.325207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:44.325207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:44.326161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:44.326161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:44.326161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:44.326161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:44.326161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:44.326161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:44.326161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:44.326161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:44.326161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 02 08:56:44.326161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:44.326161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 417/807] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:56:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:44.326161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbdfaf-1821-40de-8a8f-3a77655e025b None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:44.406053 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33687937-8654-4ab0-b0e5-c35fe64eab89 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:44.432255 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbdfaf-1821-40de-8a8f-3a77655e025b None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:44.441332 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894141d3-2cb5-4474-ac86-ca667240a76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:44.441959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894141d3-2cb5-4474-ac86-ca667240a76e None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:44.442239 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894141d3-2cb5-4474-ac86-ca667240a76e None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:44.442504 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894141d3-2cb5-4474-ac86-ca667240a76e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:44.450156 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894141d3-2cb5-4474-ac86-ca667240a76e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:44.451632 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894141d3-2cb5-4474-ac86-ca667240a76e None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:44.463797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-894141d3-2cb5-4474-ac86-ca667240a76e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 08:56:44.463797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:44.463797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:44.463797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:44.463797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:44.463797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:44.463797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:44.463797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:44.463797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:44.463797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:44.463797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:44.463797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:44.463797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:44.463797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:44.463797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:44.463797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:44.464858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:44.464858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:44.464858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:44.464858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:44.464858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:44.464858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:44.464858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:44.464858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:44.464858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:44.464858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:44.464858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:44.464858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:44.464858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:44.464858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:44.464858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:44.464858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:44.464858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:44.464858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:44.465846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:44.465846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:44.465846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:44.465846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:44.465846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:44.465846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:44.465846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:44.465846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:44.465846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 08:56:44.465846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:44.465846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 391/808] 10.222.0.36 () {56 vars in 1117 bytes} [Fri Sep 2 08:56:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:44.465846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894141d3-2cb5-4474-ac86-ca667240a76e None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:44.470829 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:44.475271 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf51fc62-88f9-47e0-a73c-4beccb6bddb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:44.476115 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51fc62-88f9-47e0-a73c-4beccb6bddb6 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:44.476547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51fc62-88f9-47e0-a73c-4beccb6bddb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:44.477016 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51fc62-88f9-47e0-a73c-4beccb6bddb6 None admin] PATH_INFO: `/v3/services/network` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:44.477606 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf51fc62-88f9-47e0-a73c-4beccb6bddb6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:44.478977 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf51fc62-88f9-47e0-a73c-4beccb6bddb6 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:44.488651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-cf51fc62-88f9-47e0-a73c-4beccb6bddb6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 08:56:44.488651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:44.488651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:56:44.488651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:56:44.488651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:56:44.488651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:56:44.488651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:56:44.488651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:44.488651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 08:56:44.488651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:44.488651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:56:44.488651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:44.488651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:44.488651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:44.488651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:44.488651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:44.488651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:44.489618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:44.489618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:44.489618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:44.489618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:44.489618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:44.489618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:44.489618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:56:44.489618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:56:44.489618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:56:44.489618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:56:44.489618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:44.489618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:44.489618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:44.489618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:44.489618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:44.489618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:44.490490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:44.490490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:44.490490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:44.490490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:44.490490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:44.490490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:44.490490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:44.490490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:44.490490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:44.490490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:44.490490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:56:44.490490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:44.490490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 08:56:44.490490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:44.494546 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 418/809] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:44 2022] GET /identity/v3/services/network => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:44.495427 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf51fc62-88f9-47e0-a73c-4beccb6bddb6 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:44.531852 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:44.576525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894141d3-2cb5-4474-ac86-ca667240a76e None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:44.605089 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf51fc62-88f9-47e0-a73c-4beccb6bddb6 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:44.613524 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519f8f2-b3f2-4be5-b99c-64f76d61224e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:44.614219 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519f8f2-b3f2-4be5-b99c-64f76d61224e None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:44.614566 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519f8f2-b3f2-4be5-b99c-64f76d61224e None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:44.614908 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519f8f2-b3f2-4be5-b99c-64f76d61224e None admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:44.615708 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519f8f2-b3f2-4be5-b99c-64f76d61224e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:44.616077 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519f8f2-b3f2-4be5-b99c-64f76d61224e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:44.617559 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519f8f2-b3f2-4be5-b99c-64f76d61224e None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:44.635468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 392/810] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:44.635468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519f8f2-b3f2-4be5-b99c-64f76d61224e None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:44.646120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe9fcf-2a7d-48e6-86c2-44a8f31d96d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:44.646718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe9fcf-2a7d-48e6-86c2-44a8f31d96d8 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:44.646991 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe9fcf-2a7d-48e6-86c2-44a8f31d96d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:44.647293 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe9fcf-2a7d-48e6-86c2-44a8f31d96d8 None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:44.647998 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe9fcf-2a7d-48e6-86c2-44a8f31d96d8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:44.648340 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe9fcf-2a7d-48e6-86c2-44a8f31d96d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:44.649704 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe9fcf-2a7d-48e6-86c2-44a8f31d96d8 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:44.672094 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 419/811] 10.222.0.36 () {56 vars in 1135 bytes} [Fri Sep 2 08:56:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:44.672094 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe9fcf-2a7d-48e6-86c2-44a8f31d96d8 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:44.703330 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:44.744167 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519f8f2-b3f2-4be5-b99c-64f76d61224e None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:44.768554 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:44.778110 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74b090-f6aa-4995-b182-14e920926eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:44.778713 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74b090-f6aa-4995-b182-14e920926eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:44.778986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74b090-f6aa-4995-b182-14e920926eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:44.779279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa74b090-f6aa-4995-b182-14e920926eb4 None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:44.780001 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74b090-f6aa-4995-b182-14e920926eb4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:44.780301 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74b090-f6aa-4995-b182-14e920926eb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:44.781218 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe9fcf-2a7d-48e6-86c2-44a8f31d96d8 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:44.781702 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa74b090-f6aa-4995-b182-14e920926eb4 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:44.798178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 393/812] 10.222.0.36 () {56 vars in 1149 bytes} [Fri Sep 2 08:56:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:44.799680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74b090-f6aa-4995-b182-14e920926eb4 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:44.820668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b544ac1-0872-40c6-a410-839b5f7d24f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:44.821266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b544ac1-0872-40c6-a410-839b5f7d24f7 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:44.821542 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b544ac1-0872-40c6-a410-839b5f7d24f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:44.821810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b544ac1-0872-40c6-a410-839b5f7d24f7 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:44.822431 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b544ac1-0872-40c6-a410-839b5f7d24f7 None admin] RBAC: Adding query filter params (name=network) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:44.822729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b544ac1-0872-40c6-a410-839b5f7d24f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:44.824121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b544ac1-0872-40c6-a410-839b5f7d24f7 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:44.828446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 420/813] 10.222.0.36 () {56 vars in 1112 bytes} [Fri Sep 2 08:56:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:44.829208 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b544ac1-0872-40c6-a410-839b5f7d24f7 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:44.896874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa74b090-f6aa-4995-b182-14e920926eb4 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:44.918105 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:44.929639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b658702-13aa-4d3d-8ec0-1662140519e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:44.930231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b658702-13aa-4d3d-8ec0-1662140519e5 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:44.930509 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b658702-13aa-4d3d-8ec0-1662140519e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:44.930779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b658702-13aa-4d3d-8ec0-1662140519e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:44.931524 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b658702-13aa-4d3d-8ec0-1662140519e5 None admin] RBAC: Adding query filter params (role.id=a0a76de72fba4036bc1e3e64df717c7f, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=cce502a83f5343558c2136c0193e7d85) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:44.931621 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b544ac1-0872-40c6-a410-839b5f7d24f7 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:44.932061 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b658702-13aa-4d3d-8ec0-1662140519e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:44.933220 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b658702-13aa-4d3d-8ec0-1662140519e5 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:44.938972 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 394/814] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:56:44 2022] GET /identity/v3/role_assignments?user.id=cce502a83f5343558c2136c0193e7d85&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=a0a76de72fba4036bc1e3e64df717c7f => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:44.940717 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b82fd-a80c-4450-a544-32568e352678 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:44.941016 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b82fd-a80c-4450-a544-32568e352678 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:44.941273 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b82fd-a80c-4450-a544-32568e352678 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:44.942755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 395/815] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:44.943495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b82fd-a80c-4450-a544-32568e352678 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:44.970433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4e9a5-ca40-4557-87cd-c2ab3579f478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:44.971645 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4e9a5-ca40-4557-87cd-c2ab3579f478 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:44.971932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4e9a5-ca40-4557-87cd-c2ab3579f478 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:44.972204 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4e9a5-ca40-4557-87cd-c2ab3579f478 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:44.983802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4e9a5-ca40-4557-87cd-c2ab3579f478 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:44.985272 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4e9a5-ca40-4557-87cd-c2ab3579f478 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:45.002667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-8aa4e9a5-ca40-4557-87cd-c2ab3579f478 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:45.002667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:45.002667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:45.002667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:45.002667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:45.002667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:45.002667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:45.002667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:45.002667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:45.002667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:45.002667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:45.002667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:45.002667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:45.002667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:45.002667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:45.002667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:45.003881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:45.003881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:45.003881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:45.003881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:45.003881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:45.003881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:45.003881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:45.003881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:45.003881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:45.003881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:45.003881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:45.003881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:45.003881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:45.003881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:45.003881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:45.003881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:45.003881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:45.005304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:45.005304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:45.005304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:45.005304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 421/816] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:45.005304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4e9a5-ca40-4557-87cd-c2ab3579f478 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:45.009484 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:45.079152 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b82fd-a80c-4450-a544-32568e352678 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:45.120153 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4e9a5-ca40-4557-87cd-c2ab3579f478 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:45.124361 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b60cc-aacc-46af-9377-5c64e69c6b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:45.125495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b60cc-aacc-46af-9377-5c64e69c6b49 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:45.125918 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b60cc-aacc-46af-9377-5c64e69c6b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:45.126347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3b60cc-aacc-46af-9377-5c64e69c6b49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:45.138365 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b60cc-aacc-46af-9377-5c64e69c6b49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:45.140291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3b60cc-aacc-46af-9377-5c64e69c6b49 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:45.156688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dc92b8-b680-4cb5-a9d0-4d501fa85232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:45.157300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc92b8-b680-4cb5-a9d0-4d501fa85232 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:45.157584 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc92b8-b680-4cb5-a9d0-4d501fa85232 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:45.157858 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dc92b8-b680-4cb5-a9d0-4d501fa85232 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:45.158486 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc92b8-b680-4cb5-a9d0-4d501fa85232 None admin] RBAC: Adding query filter params (type=network) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:45.159625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc92b8-b680-4cb5-a9d0-4d501fa85232 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:45.160649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-db3b60cc-aacc-46af-9377-5c64e69c6b49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:45.160649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:45.160649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:45.160649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:45.160649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:45.160649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:45.160649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:45.160649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:45.160649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:45.160649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:45.160649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:45.160649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:45.160649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:45.160649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:45.160649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:45.160649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:45.161815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:45.161815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:45.161815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:45.161815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:45.161815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:45.161815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:45.161815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:45.161815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:45.161815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:45.161815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:45.161815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:45.161815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:45.161815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:45.161815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:45.161815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:45.161815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:45.161815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:45.162850 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:45.162850 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:45.162850 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:45.163020 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dc92b8-b680-4cb5-a9d0-4d501fa85232 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:45.169042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 422/818] 10.222.0.36 () {56 vars in 1112 bytes} [Fri Sep 2 08:56:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:45.169042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11b0ea-561c-4f34-befc-522c3a888ef5 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:45.169042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11b0ea-561c-4f34-befc-522c3a888ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:45.169042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11b0ea-561c-4f34-befc-522c3a888ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:45.169439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 396/818] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:45.170509 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b60cc-aacc-46af-9377-5c64e69c6b49 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:45.187529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-5d11b0ea-561c-4f34-befc-522c3a888ef5 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:45.188751 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5d11b0ea-561c-4f34-befc-522c3a888ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:45.256424 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:45.267110 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 423/819] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:56:45.267110 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11b0ea-561c-4f34-befc-522c3a888ef5 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:45.347297 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:45.389668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11b0ea-561c-4f34-befc-522c3a888ef5 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:45.426077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5dd84c-b312-4a72-b36a-90f47dc9d528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:45.426984 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5dd84c-b312-4a72-b36a-90f47dc9d528 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:45.427299 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5dd84c-b312-4a72-b36a-90f47dc9d528 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:45.427576 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5dd84c-b312-4a72-b36a-90f47dc9d528 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:45.428424 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5dd84c-b312-4a72-b36a-90f47dc9d528 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:45.428578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5dd84c-b312-4a72-b36a-90f47dc9d528 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:45.429957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5dd84c-b312-4a72-b36a-90f47dc9d528 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:45.444964 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3b60cc-aacc-46af-9377-5c64e69c6b49 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:45.450003 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 424/820] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:45.450003 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5dd84c-b312-4a72-b36a-90f47dc9d528 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:45.486350 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0264ee2-9447-4a99-8bf6-60725bc26777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:45.487435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0264ee2-9447-4a99-8bf6-60725bc26777 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:45.487790 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0264ee2-9447-4a99-8bf6-60725bc26777 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:45.488131 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0264ee2-9447-4a99-8bf6-60725bc26777 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:45.488848 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0264ee2-9447-4a99-8bf6-60725bc26777 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:45.489221 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0264ee2-9447-4a99-8bf6-60725bc26777 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:45.490655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0264ee2-9447-4a99-8bf6-60725bc26777 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:45.507450 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 397/821] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:45.508366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0264ee2-9447-4a99-8bf6-60725bc26777 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:45.525136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:45.573822 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5dd84c-b312-4a72-b36a-90f47dc9d528 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:45.613349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4cea4-3587-4b63-a70b-b3eae113a205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:45.613957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4cea4-3587-4b63-a70b-b3eae113a205 None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:45.614232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4cea4-3587-4b63-a70b-b3eae113a205 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:45.614502 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4cea4-3587-4b63-a70b-b3eae113a205 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:45.615211 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af4cea4-3587-4b63-a70b-b3eae113a205 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:45.616569 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af4cea4-3587-4b63-a70b-b3eae113a205 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:45.617635 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:45.621761 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0264ee2-9447-4a99-8bf6-60725bc26777 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:45.651099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 425/822] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 08:56:45 2022] POST /identity/v3/endpoints => generated 337 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:56:45.651099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4cea4-3587-4b63-a70b-b3eae113a205 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:45.681321 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4397e-7bdf-48bf-97fe-205fba4a5ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:45.682332 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4397e-7bdf-48bf-97fe-205fba4a5ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:45.682679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4397e-7bdf-48bf-97fe-205fba4a5ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:45.683162 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4397e-7bdf-48bf-97fe-205fba4a5ad0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:45.694235 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4397e-7bdf-48bf-97fe-205fba4a5ad0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:45.696722 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4397e-7bdf-48bf-97fe-205fba4a5ad0 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:45.715541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-f4e4397e-7bdf-48bf-97fe-205fba4a5ad0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:45.715541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:45.715541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:45.715541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:45.715541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:45.715541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:45.715541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:45.715541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:45.715541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:45.715541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:45.715541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:45.715541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:45.715541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:45.715541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:45.715541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:45.716814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:45.716814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:45.716814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:45.716814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:45.716814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:45.716814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:45.716814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:45.716814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:45.716814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:45.716814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:45.716814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:45.716814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:45.716814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:45.716814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:45.716814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:45.716814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:45.716814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:45.718009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:45.718009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:45.718009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:45.718009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:45.718009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:45.718009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:45.718009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:45.718009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:45.723281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 398/823] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:45.724467 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4397e-7bdf-48bf-97fe-205fba4a5ad0 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:45.781652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4cea4-3587-4b63-a70b-b3eae113a205 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:45.797009 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:45.829992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24573823-3a00-4110-bd76-f4ac27a1595e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:45.830664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24573823-3a00-4110-bd76-f4ac27a1595e None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:45.830948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24573823-3a00-4110-bd76-f4ac27a1595e None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:45.831263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24573823-3a00-4110-bd76-f4ac27a1595e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:45.838945 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24573823-3a00-4110-bd76-f4ac27a1595e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:45.840425 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24573823-3a00-4110-bd76-f4ac27a1595e None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:45.853989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-24573823-3a00-4110-bd76-f4ac27a1595e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:45.853989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:45.853989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:45.853989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:45.853989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:45.853989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:45.853989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:45.853989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:45.853989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:45.853989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:45.853989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:45.853989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:45.853989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:45.853989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:45.853989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:45.854935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:45.854935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:45.854935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:45.854935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:45.854935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:45.854935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:45.854935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:45.854935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:45.854935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:45.854935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:45.854935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:45.854935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:45.854935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:45.854935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:45.854935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:45.854935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:45.854935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:45.855877 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:45.855877 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:45.855877 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:45.855877 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:45.855877 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:45.855877 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:45.855877 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:45.855877 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:45.856314 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 426/824] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:45.857501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24573823-3a00-4110-bd76-f4ac27a1595e None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:45.876681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4397e-7bdf-48bf-97fe-205fba4a5ad0 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:45.918065 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8671b-a2ab-45d1-b7a8-3c1884d4bc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:45.918699 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8671b-a2ab-45d1-b7a8-3c1884d4bc62 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:45.918974 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8671b-a2ab-45d1-b7a8-3c1884d4bc62 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:45.919277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8671b-a2ab-45d1-b7a8-3c1884d4bc62 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:45.919862 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8671b-a2ab-45d1-b7a8-3c1884d4bc62 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:45.921332 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d8671b-a2ab-45d1-b7a8-3c1884d4bc62 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:45.930852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-42d8671b-a2ab-45d1-b7a8-3c1884d4bc62 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 08:56:45.930852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:45.930852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:56:45.930852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:56:45.930852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:56:45.930852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:56:45.930852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:56:45.930852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:45.930852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 08:56:45.930852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:45.930852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:56:45.930852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:45.930852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:45.930852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:45.930852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:45.930852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:45.930852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:45.931932 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:45.931932 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:45.931932 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:45.931932 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:45.931932 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:45.931932 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:45.931932 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:56:45.931932 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:56:45.931932 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:56:45.931932 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:56:45.931932 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:45.931932 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:45.931932 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:45.931932 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:45.931932 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:45.931932 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:45.933196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:45.933196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:45.933196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:45.933196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:45.933196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:45.933196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:45.933196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:45.933196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:45.933196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:45.933196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:45.933196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:56:45.933196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:45.933196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 08:56:45.933196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:45.937215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 399/825] 10.222.0.36 () {56 vars in 1129 bytes} [Fri Sep 2 08:56:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:45.938134 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8671b-a2ab-45d1-b7a8-3c1884d4bc62 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:45.959257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:45.978054 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24573823-3a00-4110-bd76-f4ac27a1595e None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:46.013460 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade655b-fe1b-4cab-b7ba-87edb78640b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:46.014068 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade655b-fe1b-4cab-b7ba-87edb78640b6 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:46.014353 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade655b-fe1b-4cab-b7ba-87edb78640b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:46.014624 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade655b-fe1b-4cab-b7ba-87edb78640b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:46.015332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade655b-fe1b-4cab-b7ba-87edb78640b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:46.015638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade655b-fe1b-4cab-b7ba-87edb78640b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:46.017062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade655b-fe1b-4cab-b7ba-87edb78640b6 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:46.035378 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 427/826] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:46.035976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade655b-fe1b-4cab-b7ba-87edb78640b6 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:46.045778 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8671b-a2ab-45d1-b7a8-3c1884d4bc62 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:46.072520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:46.085360 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558d24b-a73e-47c5-be17-c1204cea878d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:46.086063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558d24b-a73e-47c5-be17-c1204cea878d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:46.086350 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558d24b-a73e-47c5-be17-c1204cea878d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:46.086626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e558d24b-a73e-47c5-be17-c1204cea878d None admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:46.087352 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558d24b-a73e-47c5-be17-c1204cea878d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:46.087662 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558d24b-a73e-47c5-be17-c1204cea878d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:46.089092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e558d24b-a73e-47c5-be17-c1204cea878d None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:46.105874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 400/827] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:46.105874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558d24b-a73e-47c5-be17-c1204cea878d None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:46.153976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade655b-fe1b-4cab-b7ba-87edb78640b6 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:46.188969 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0c08a-2a74-482d-8979-3aea3e16a9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:46.189485 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:46.189971 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0c08a-2a74-482d-8979-3aea3e16a9bf None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:46.189971 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0c08a-2a74-482d-8979-3aea3e16a9bf None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:46.190312 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b0c08a-2a74-482d-8979-3aea3e16a9bf None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:46.191174 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b0c08a-2a74-482d-8979-3aea3e16a9bf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:46.191517 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b0c08a-2a74-482d-8979-3aea3e16a9bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:46.192912 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b0c08a-2a74-482d-8979-3aea3e16a9bf None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:46.201840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 428/828] 10.222.0.36 () {56 vars in 1124 bytes} [Fri Sep 2 08:56:46 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:46.201840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0c08a-2a74-482d-8979-3aea3e16a9bf None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:46.218199 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e558d24b-a73e-47c5-be17-c1204cea878d None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:46.252360 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f696cb-beca-460a-8e7f-344ba1abb91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:46.253017 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f696cb-beca-460a-8e7f-344ba1abb91a None admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:46.253297 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f696cb-beca-460a-8e7f-344ba1abb91a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:46.253571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f696cb-beca-460a-8e7f-344ba1abb91a None admin] PATH_INFO: `/v3/projects/eef80493db09437b88f2b147a59928c1/users/76ebecce283f491e82a961d7f739878f/roles/637ed3cd3d484839ab21de02c2bbe293` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:46.268664 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f696cb-beca-460a-8e7f-344ba1abb91a None admin] RBAC: Authorizing `identity:create_grant(project_id=eef80493db09437b88f2b147a59928c1, user_id=76ebecce283f491e82a961d7f739878f, role_id=637ed3cd3d484839ab21de02c2bbe293)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:46.270094 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f696cb-beca-460a-8e7f-344ba1abb91a None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:46.284909 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 401/829] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:56:46 2022] PUT /identity/v3/projects/eef80493db09437b88f2b147a59928c1/users/76ebecce283f491e82a961d7f739878f/roles/637ed3cd3d484839ab21de02c2bbe293 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:56:46.285757 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f696cb-beca-460a-8e7f-344ba1abb91a None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:46.316774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:46.323056 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b0c08a-2a74-482d-8979-3aea3e16a9bf None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:46.372795 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c31c0-3c6e-448f-b9c7-9ce9d24a79cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:46.373487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c31c0-3c6e-448f-b9c7-9ce9d24a79cb None admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:46.373768 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c31c0-3c6e-448f-b9c7-9ce9d24a79cb None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:46.374113 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c31c0-3c6e-448f-b9c7-9ce9d24a79cb None admin] PATH_INFO: `/v3/projects/eef80493db09437b88f2b147a59928c1/users/92318948dcf1407b9fc8aa5c199787ae/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:46.377047 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c31c0-3c6e-448f-b9c7-9ce9d24a79cb None admin] RBAC: Authorizing `identity:create_grant(project_id=eef80493db09437b88f2b147a59928c1, user_id=92318948dcf1407b9fc8aa5c199787ae, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:46.379344 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288c31c0-3c6e-448f-b9c7-9ce9d24a79cb None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:46.407103 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 429/830] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:56:46 2022] PUT /identity/v3/projects/eef80493db09437b88f2b147a59928c1/users/92318948dcf1407b9fc8aa5c199787ae/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:56:46.433953 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f696cb-beca-460a-8e7f-344ba1abb91a None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:46.473574 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:46.481619 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740db7b-3950-45d8-8373-3fee080fd303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:46.482289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740db7b-3950-45d8-8373-3fee080fd303 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:46.482605 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740db7b-3950-45d8-8373-3fee080fd303 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:46.483864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740db7b-3950-45d8-8373-3fee080fd303 None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:46.483864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740db7b-3950-45d8-8373-3fee080fd303 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:46.484195 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740db7b-3950-45d8-8373-3fee080fd303 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:46.485981 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740db7b-3950-45d8-8373-3fee080fd303 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:46.492405 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 402/831] 10.222.0.36 () {56 vars in 1124 bytes} [Fri Sep 2 08:56:46 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:46.498222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c31c0-3c6e-448f-b9c7-9ce9d24a79cb None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:46.615370 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c31c0-3c6e-448f-b9c7-9ce9d24a79cb None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:46.651722 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ed152-5614-4593-8269-da009111262e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:46.652480 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ed152-5614-4593-8269-da009111262e None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:46.652847 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ed152-5614-4593-8269-da009111262e None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:46.653213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ed152-5614-4593-8269-da009111262e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:46.653989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ed152-5614-4593-8269-da009111262e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:46.655511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ed152-5614-4593-8269-da009111262e None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:46.681763 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 430/832] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 08:56:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:56:46.909903 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d625b6-e4b7-43cc-918c-a109518de346 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:46.910253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d625b6-e4b7-43cc-918c-a109518de346 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:46.910358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d625b6-e4b7-43cc-918c-a109518de346 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:46.911931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 403/833] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:46.917202 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57013a8-b888-4a9c-8b0a-728a77eb99a7 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:46.917488 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57013a8-b888-4a9c-8b0a-728a77eb99a7 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:46.917758 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57013a8-b888-4a9c-8b0a-728a77eb99a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:46.936684 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-b57013a8-b888-4a9c-8b0a-728a77eb99a7 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:46.937859 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-b57013a8-b888-4a9c-8b0a-728a77eb99a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:46.949076 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:47.015886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 431/834] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:46 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:56:47.282638 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d625b6-e4b7-43cc-918c-a109518de346 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:47.435097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d625b6-e4b7-43cc-918c-a109518de346 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:47.478485 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9951292-9fa3-4d57-b0d3-b2084f376f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:47.479165 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9951292-9fa3-4d57-b0d3-b2084f376f3c None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:47.479444 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9951292-9fa3-4d57-b0d3-b2084f376f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:47.479714 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9951292-9fa3-4d57-b0d3-b2084f376f3c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:47.480300 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9951292-9fa3-4d57-b0d3-b2084f376f3c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:47.481939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9951292-9fa3-4d57-b0d3-b2084f376f3c None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:47.492060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-b9951292-9fa3-4d57-b0d3-b2084f376f3c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 08:56:47.492060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:47.492060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:56:47.492060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:56:47.492060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:56:47.492060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:56:47.492060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:56:47.492060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:47.492060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 08:56:47.492060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:47.492060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:56:47.492060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:47.492060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:47.492060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:47.492060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:47.492060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:47.493135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:47.493135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:47.493135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:47.493135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:47.493135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:47.493135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:47.493135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:47.493135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:56:47.493135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:56:47.493135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:56:47.493135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:56:47.493135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:47.493135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:47.493135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:47.493135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:47.493135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:47.493135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:47.494178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:47.494178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:47.494178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:47.494178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:47.494178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:47.494178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:47.494178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:47.494178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:47.494178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:47.494178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:47.494178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:56:47.494178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:47.494178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 08:56:47.494178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:47.494178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 404/835] 10.222.0.36 () {56 vars in 1132 bytes} [Fri Sep 2 08:56:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:47.500932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57013a8-b888-4a9c-8b0a-728a77eb99a7 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:47.604628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57013a8-b888-4a9c-8b0a-728a77eb99a7 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:47.640279 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780cb66-e89e-4075-992a-2d97ed17813e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:47.641132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780cb66-e89e-4075-992a-2d97ed17813e None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:47.641720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780cb66-e89e-4075-992a-2d97ed17813e None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:47.641992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780cb66-e89e-4075-992a-2d97ed17813e None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:47.643380 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780cb66-e89e-4075-992a-2d97ed17813e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:47.644245 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780cb66-e89e-4075-992a-2d97ed17813e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:47.645844 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780cb66-e89e-4075-992a-2d97ed17813e None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:47.651980 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 432/836] 10.222.0.36 () {56 vars in 1126 bytes} [Fri Sep 2 08:56:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:47.657457 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9951292-9fa3-4d57-b0d3-b2084f376f3c None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:47.732917 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:47.779182 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9951292-9fa3-4d57-b0d3-b2084f376f3c None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:47.813722 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3e71d-89c7-4cfd-891b-6c7f9ac6f995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:47.814335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3e71d-89c7-4cfd-891b-6c7f9ac6f995 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:47.814645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3e71d-89c7-4cfd-891b-6c7f9ac6f995 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:47.815617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3e71d-89c7-4cfd-891b-6c7f9ac6f995 None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:47.816246 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3e71d-89c7-4cfd-891b-6c7f9ac6f995 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:47.816546 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3e71d-89c7-4cfd-891b-6c7f9ac6f995 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:47.817905 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be3e71d-89c7-4cfd-891b-6c7f9ac6f995 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:47.827236 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 405/837] 10.222.0.36 () {56 vars in 1126 bytes} [Fri Sep 2 08:56:47 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:48.360734 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53dcf6-a58e-4c60-86c5-0bbc2349d975 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:48.361165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53dcf6-a58e-4c60-86c5-0bbc2349d975 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:48.361355 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53dcf6-a58e-4c60-86c5-0bbc2349d975 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:48.363473 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 433/838] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:48.368686 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6b1db-d421-4b1e-8b31-da08b5e4849c None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:48.369096 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6b1db-d421-4b1e-8b31-da08b5e4849c None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:48.369264 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6b1db-d421-4b1e-8b31-da08b5e4849c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:48.388175 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-4df6b1db-d421-4b1e-8b31-da08b5e4849c None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:48.389379 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-4df6b1db-d421-4b1e-8b31-da08b5e4849c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:48.461671 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 406/839] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:56:48.477728 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f57d5f-5cbf-4e65-8307-3a36cdc3f7e7 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:48.478026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f57d5f-5cbf-4e65-8307-3a36cdc3f7e7 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:48.478283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f57d5f-5cbf-4e65-8307-3a36cdc3f7e7 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:48.479910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 434/840] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:48.488777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6bcc5-94bd-4c1b-af19-c66d164f1b57 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:48.489055 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6bcc5-94bd-4c1b-af19-c66d164f1b57 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:48.489328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6bcc5-94bd-4c1b-af19-c66d164f1b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:48.509850 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-0aa6bcc5-94bd-4c1b-af19-c66d164f1b57 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:48.511023 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-0aa6bcc5-94bd-4c1b-af19-c66d164f1b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:48.558077 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:48.580350 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 407/841] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:56:48.739724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f57d5f-5cbf-4e65-8307-3a36cdc3f7e7 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:48.743647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:48.848154 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f57d5f-5cbf-4e65-8307-3a36cdc3f7e7 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:48.855654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6bcc5-94bd-4c1b-af19-c66d164f1b57 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:48.896125 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679c0d0-ccbe-473e-a2fd-0902a7942169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:48.896817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679c0d0-ccbe-473e-a2fd-0902a7942169 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:48.897099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679c0d0-ccbe-473e-a2fd-0902a7942169 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:48.897363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679c0d0-ccbe-473e-a2fd-0902a7942169 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:48.910674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679c0d0-ccbe-473e-a2fd-0902a7942169 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:48.915512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-e679c0d0-ccbe-473e-a2fd-0902a7942169 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:48.915512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:48.915512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:48.915512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:48.915512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:48.915512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:48.915512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:48.915512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:48.915512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:48.915512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:48.915512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:48.915512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:48.915512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:48.915512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:48.915512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:48.915512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:48.916441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:48.916441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:48.916441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:48.916441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:48.916441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:48.916441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:48.916441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:48.916441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:48.916441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:48.916441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:48.916441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:48.916441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:48.916441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:48.916441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:48.916441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:48.916441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:48.917369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:48.917369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:48.917369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:48.917369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:48.917369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:48.917369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:48.917369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:48.917369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:48.917369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:48.917369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:48.917369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:56:48.917369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:48.917369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 435/842] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:56:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:48.918091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425dc05-9d0b-4479-ae59-66a3b39e95ed None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:48.918091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425dc05-9d0b-4479-ae59-66a3b39e95ed None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:48.918741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425dc05-9d0b-4479-ae59-66a3b39e95ed None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:48.919523 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 436/843] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:48.924113 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425dc05-9d0b-4479-ae59-66a3b39e95ed None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:48.963708 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6bcc5-94bd-4c1b-af19-c66d164f1b57 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:48.998259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:49.003513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a2d83-91b8-44c7-987a-f31c389fd4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:49.004139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a2d83-91b8-44c7-987a-f31c389fd4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:49.004412 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a2d83-91b8-44c7-987a-f31c389fd4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:49.004679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a2d83-91b8-44c7-987a-f31c389fd4c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:49.015556 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a2d83-91b8-44c7-987a-f31c389fd4c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:49.020329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-506a2d83-91b8-44c7-987a-f31c389fd4c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:56:49.020329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:49.020329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:49.020329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:49.020329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:49.020329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:49.020329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:49.020329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:49.020329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:49.020329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:49.020329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:49.020329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:49.020329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:49.020329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:49.020329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:49.020329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:49.021263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:49.021263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:49.021263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:49.021263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:49.021263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:49.021263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:49.021263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:49.021263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:49.021263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:49.021263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:49.021263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:49.021263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:49.021263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:49.021263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:49.021263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:49.021263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:49.022123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:49.022123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:49.022123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:49.022123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:49.022123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:49.022123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:49.022123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:49.022123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:49.022123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:49.022123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:49.022123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:56:49.022123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:49.026328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 408/844] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:56:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:49.027556 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8ca72-757b-4acb-a77d-e6e10f6a06ac None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:49.027838 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8ca72-757b-4acb-a77d-e6e10f6a06ac None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:49.028111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8ca72-757b-4acb-a77d-e6e10f6a06ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:49.043419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425dc05-9d0b-4479-ae59-66a3b39e95ed None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:49.046460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-7fe8ca72-757b-4acb-a77d-e6e10f6a06ac None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:49.047645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7fe8ca72-757b-4acb-a77d-e6e10f6a06ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:49.083460 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546c16a-d325-44eb-9c01-8e3bbf11949e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:49.084060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546c16a-d325-44eb-9c01-8e3bbf11949e None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:49.084333 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546c16a-d325-44eb-9c01-8e3bbf11949e None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:49.084598 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546c16a-d325-44eb-9c01-8e3bbf11949e None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:49.085243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546c16a-d325-44eb-9c01-8e3bbf11949e None admin] RBAC: Adding query filter params (name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:49.085538 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546c16a-d325-44eb-9c01-8e3bbf11949e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:49.086889 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3546c16a-d325-44eb-9c01-8e3bbf11949e None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:49.087320 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:49.099113 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 437/845] 10.222.0.36 () {56 vars in 1103 bytes} [Fri Sep 2 08:56:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:49.099113 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546c16a-d325-44eb-9c01-8e3bbf11949e None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:49.135119 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 409/846] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:56:49.135119 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8ca72-757b-4acb-a77d-e6e10f6a06ac None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:49.215567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3546c16a-d325-44eb-9c01-8e3bbf11949e None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:49.245670 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8ca72-757b-4acb-a77d-e6e10f6a06ac None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:49.251565 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:49.257773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f916ef-3fc7-4841-b207-cf9b3a097085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:49.258414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f916ef-3fc7-4841-b207-cf9b3a097085 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:49.259005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f916ef-3fc7-4841-b207-cf9b3a097085 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:49.259315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f916ef-3fc7-4841-b207-cf9b3a097085 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:49.260259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f916ef-3fc7-4841-b207-cf9b3a097085 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:49.260566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f916ef-3fc7-4841-b207-cf9b3a097085 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:49.262474 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f916ef-3fc7-4841-b207-cf9b3a097085 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:49.275353 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 438/847] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:56:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:49.280146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f916ef-3fc7-4841-b207-cf9b3a097085 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:49.288737 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdadca19-67e6-4f36-b0d1-a537e937903d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:49.289546 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadca19-67e6-4f36-b0d1-a537e937903d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:49.289914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadca19-67e6-4f36-b0d1-a537e937903d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:49.290331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadca19-67e6-4f36-b0d1-a537e937903d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:49.301595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdadca19-67e6-4f36-b0d1-a537e937903d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:49.303215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdadca19-67e6-4f36-b0d1-a537e937903d None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:49.318364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-fdadca19-67e6-4f36-b0d1-a537e937903d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:49.318364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:49.318364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:49.318364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:49.318364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:49.318364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:49.318364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:49.318364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:49.318364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:49.318364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:49.318364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:49.318364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:49.318364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:49.318364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:49.318364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:49.318364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:49.319551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:49.319551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:49.319551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:49.319551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:49.319551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:49.319551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:49.319551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:49.319551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:49.319551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:49.319551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:49.319551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:49.319551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:49.319551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:49.319551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:49.319551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:49.319551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:49.319551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:49.320670 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:49.320670 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:49.320670 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:49.322012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 410/848] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:49.327326 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdadca19-67e6-4f36-b0d1-a537e937903d None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:49.353548 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:49.400843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f916ef-3fc7-4841-b207-cf9b3a097085 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:49.433942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdadca19-67e6-4f36-b0d1-a537e937903d None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:49.440642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6016fd1-5854-434a-87f7-dff8a3de073b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:49.441659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6016fd1-5854-434a-87f7-dff8a3de073b None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:49.441949 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6016fd1-5854-434a-87f7-dff8a3de073b None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:49.442220 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6016fd1-5854-434a-87f7-dff8a3de073b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:49.453369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6016fd1-5854-434a-87f7-dff8a3de073b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:49.454807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6016fd1-5854-434a-87f7-dff8a3de073b None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:49.468858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031a2b2-9972-45f7-bf88-5ae364f9ab58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:49.469450 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031a2b2-9972-45f7-bf88-5ae364f9ab58 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:49.469723 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031a2b2-9972-45f7-bf88-5ae364f9ab58 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:49.469976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-e6016fd1-5854-434a-87f7-dff8a3de073b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:49.469976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:49.469976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:49.469976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:49.469976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:49.469976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:49.469976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:49.469976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:49.469976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:49.469976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:49.469976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:49.469976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:49.469976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:49.469976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:49.469976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:49.469976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:49.470948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:49.470948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:49.470948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:49.470948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:49.470948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:49.470948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:49.470948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:49.470948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:49.470948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:49.470948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:49.470948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:49.470948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:49.470948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:49.470948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:49.470948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:49.470948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:49.470948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:49.471898 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:49.471898 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:49.471898 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:49.472072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031a2b2-9972-45f7-bf88-5ae364f9ab58 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:49.472072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031a2b2-9972-45f7-bf88-5ae364f9ab58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:49.472072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031a2b2-9972-45f7-bf88-5ae364f9ab58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:49.472706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031a2b2-9972-45f7-bf88-5ae364f9ab58 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:49.475237 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 439/849] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:49.480415 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6016fd1-5854-434a-87f7-dff8a3de073b None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:49.485021 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 411/850] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:49.487261 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031a2b2-9972-45f7-bf88-5ae364f9ab58 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:49.533907 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:49.572313 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:49.592613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031a2b2-9972-45f7-bf88-5ae364f9ab58 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:49.605131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6016fd1-5854-434a-87f7-dff8a3de073b None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:49.628738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b974b7-13d7-48fb-b77f-4c6ce9d63166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:49.629358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b974b7-13d7-48fb-b77f-4c6ce9d63166 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:49.629645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b974b7-13d7-48fb-b77f-4c6ce9d63166 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:49.629914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b974b7-13d7-48fb-b77f-4c6ce9d63166 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:49.630556 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b974b7-13d7-48fb-b77f-4c6ce9d63166 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:49.630861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b974b7-13d7-48fb-b77f-4c6ce9d63166 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:49.632269 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b974b7-13d7-48fb-b77f-4c6ce9d63166 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:49.641649 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c38826-a94d-486a-96ac-bafa1fc7fcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:49.642441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c38826-a94d-486a-96ac-bafa1fc7fcec None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:49.643510 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c38826-a94d-486a-96ac-bafa1fc7fcec None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:49.643845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 412/851] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:49.644104 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c38826-a94d-486a-96ac-bafa1fc7fcec None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:49.644630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b974b7-13d7-48fb-b77f-4c6ce9d63166 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:49.644957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c38826-a94d-486a-96ac-bafa1fc7fcec None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:49.646322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c38826-a94d-486a-96ac-bafa1fc7fcec None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:49.657252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-35c38826-a94d-486a-96ac-bafa1fc7fcec None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 08:56:49.657252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:49.657252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:56:49.657252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:56:49.657252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:56:49.657252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:56:49.657252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:56:49.657252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:49.657252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 08:56:49.657252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:49.657252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:56:49.657252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:49.657252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:49.657252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:49.657252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:49.657252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:49.657252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:49.658266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:49.658266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:49.658266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:49.658266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:49.658266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:49.658266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:49.658266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:56:49.658266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:56:49.658266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:56:49.658266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:56:49.658266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:49.658266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:49.658266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:49.658266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:49.658266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:49.658266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:49.659198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:49.659198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:49.659198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:49.659198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:49.659198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:49.659198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:49.659198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:49.659198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:49.659198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:49.659198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:49.659198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:56:49.659198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:49.659198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 08:56:49.659198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:49.659198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 440/852] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:56:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:49.660000 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c38826-a94d-486a-96ac-bafa1fc7fcec None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:49.756850 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b974b7-13d7-48fb-b77f-4c6ce9d63166 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:49.763228 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c38826-a94d-486a-96ac-bafa1fc7fcec None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:49.765514 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:49.792888 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055546b-1531-44b1-ba73-44f2a078bc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:49.793498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055546b-1531-44b1-ba73-44f2a078bc79 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:49.793781 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055546b-1531-44b1-ba73-44f2a078bc79 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:49.794082 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055546b-1531-44b1-ba73-44f2a078bc79 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:49.798576 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1bbd2-4d65-4f26-a4a1-e0f131bbe3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:49.799751 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1bbd2-4d65-4f26-a4a1-e0f131bbe3ef None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:49.800026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1bbd2-4d65-4f26-a4a1-e0f131bbe3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:49.800289 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1bbd2-4d65-4f26-a4a1-e0f131bbe3ef None admin] PATH_INFO: `/v3/users/swift` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:49.802747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055546b-1531-44b1-ba73-44f2a078bc79 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:49.804692 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055546b-1531-44b1-ba73-44f2a078bc79 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:49.807877 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc1bbd2-4d65-4f26-a4a1-e0f131bbe3ef None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:49.809531 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc1bbd2-4d65-4f26-a4a1-e0f131bbe3ef None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:49.817913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-1055546b-1531-44b1-ba73-44f2a078bc79 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 08:56:49.817913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:49.817913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:49.817913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:49.817913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:49.817913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:49.817913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:49.817913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:49.817913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:49.817913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:49.817913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:49.817913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:49.817913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:49.817913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:49.817913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:49.817913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:49.818828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:49.818828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:49.818828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:49.818828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:49.818828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:49.818828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:49.818828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:49.818828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:49.818828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:49.818828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:49.818828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:49.818828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:49.818828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:49.818828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:49.818828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:49.818828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:49.818828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:49.818828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:49.824715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:49.824715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:49.824715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:49.824715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:49.824715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:49.824715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:49.824715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:49.824715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:49.824715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 08:56:49.824715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:49.824715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 413/853] 10.222.0.36 () {56 vars in 1117 bytes} [Fri Sep 2 08:56:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:49.824715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055546b-1531-44b1-ba73-44f2a078bc79 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:49.825363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-2cc1bbd2-4d65-4f26-a4a1-e0f131bbe3ef None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 02 08:56:49.825363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:49.825363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:49.825363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:49.825363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:49.825363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:49.825363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:49.825363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:49.825363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:49.825363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:49.825363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:49.825363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:49.825363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:49.825363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:49.825363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:49.825363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:49.826209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:49.826209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:49.826209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:49.826209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:49.826209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:49.826209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:49.826209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:49.826209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:49.826209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:49.826209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:49.826209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:49.826209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:49.826209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:49.826209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:49.826209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:49.826209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:49.826209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:49.826209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:49.828689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:49.828689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:49.828689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:49.828689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:49.828689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:49.828689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:49.828689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:49.828689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:49.828689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 02 08:56:49.828689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:49.828689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 441/854] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:56:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:49.828689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1bbd2-4d65-4f26-a4a1-e0f131bbe3ef None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:49.830120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:49.925564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1bbd2-4d65-4f26-a4a1-e0f131bbe3ef None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:49.929537 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055546b-1531-44b1-ba73-44f2a078bc79 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:49.959226 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a253c-5914-4b3a-8acc-0441ef5ee168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:49.959832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a253c-5914-4b3a-8acc-0441ef5ee168 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:49.960112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a253c-5914-4b3a-8acc-0441ef5ee168 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:49.960383 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a253c-5914-4b3a-8acc-0441ef5ee168 None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:49.961099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a253c-5914-4b3a-8acc-0441ef5ee168 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:49.961403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a253c-5914-4b3a-8acc-0441ef5ee168 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:49.962618 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbbed14-9dcd-4824-91d4-3a1256a455ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:49.962821 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a253c-5914-4b3a-8acc-0441ef5ee168 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:49.963246 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbbed14-9dcd-4824-91d4-3a1256a455ba None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:49.963522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbbed14-9dcd-4824-91d4-3a1256a455ba None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:49.963786 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbbed14-9dcd-4824-91d4-3a1256a455ba None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:49.964397 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbbed14-9dcd-4824-91d4-3a1256a455ba None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:49.964688 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbbed14-9dcd-4824-91d4-3a1256a455ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:49.966019 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbbed14-9dcd-4824-91d4-3a1256a455ba None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:49.970285 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 414/855] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:56:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:49.971059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbbed14-9dcd-4824-91d4-3a1256a455ba None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:49.983992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 442/856] 10.222.0.36 () {56 vars in 1149 bytes} [Fri Sep 2 08:56:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:49.985046 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb723161-f81e-443e-8e15-c8fd85ce732b None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:49.985330 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb723161-f81e-443e-8e15-c8fd85ce732b None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:49.985602 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb723161-f81e-443e-8e15-c8fd85ce732b None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:49.987355 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 443/857] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:49.988039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb723161-f81e-443e-8e15-c8fd85ce732b None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:50.011941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:50.039674 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:50.075928 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbbed14-9dcd-4824-91d4-3a1256a455ba None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:50.087957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb723161-f81e-443e-8e15-c8fd85ce732b None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:50.107609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559d832-0ce8-439f-b322-bde27091cad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:50.108215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559d832-0ce8-439f-b322-bde27091cad5 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:50.108493 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559d832-0ce8-439f-b322-bde27091cad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:50.108762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559d832-0ce8-439f-b322-bde27091cad5 None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:50.109477 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5559d832-0ce8-439f-b322-bde27091cad5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:50.109779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5559d832-0ce8-439f-b322-bde27091cad5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:50.111213 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5559d832-0ce8-439f-b322-bde27091cad5 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:50.120543 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeed3919-a48f-4581-a32b-f4c004852ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:50.121138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed3919-a48f-4581-a32b-f4c004852ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:50.121421 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed3919-a48f-4581-a32b-f4c004852ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:50.121692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeed3919-a48f-4581-a32b-f4c004852ee5 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:50.122311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed3919-a48f-4581-a32b-f4c004852ee5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:50.122611 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed3919-a48f-4581-a32b-f4c004852ee5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:50.123991 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeed3919-a48f-4581-a32b-f4c004852ee5 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:50.128653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 415/858] 10.222.0.36 () {56 vars in 1135 bytes} [Fri Sep 2 08:56:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:50.129090 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 444/859] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:56:49 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:50.130230 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebf51d-f1ef-4b8e-a06a-3b8986592814 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:50.130514 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebf51d-f1ef-4b8e-a06a-3b8986592814 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:50.130791 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebf51d-f1ef-4b8e-a06a-3b8986592814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:50.131864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559d832-0ce8-439f-b322-bde27091cad5 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:50.149858 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-3cebf51d-f1ef-4b8e-a06a-3b8986592814 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:50.151129 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-3cebf51d-f1ef-4b8e-a06a-3b8986592814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:50.229356 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 445/860] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:56:50.231131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cebf51d-f1ef-4b8e-a06a-3b8986592814 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:50.253471 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559d832-0ce8-439f-b322-bde27091cad5 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:50.261167 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:50.288373 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:50.304759 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e7af0-4d91-4087-8d99-84f899b439b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:50.305487 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e7af0-4d91-4087-8d99-84f899b439b1 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:50.305766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e7af0-4d91-4087-8d99-84f899b439b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:50.306037 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e7af0-4d91-4087-8d99-84f899b439b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:50.383394 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e7af0-4d91-4087-8d99-84f899b439b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:50.384834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e7af0-4d91-4087-8d99-84f899b439b1 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:50.386714 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cebf51d-f1ef-4b8e-a06a-3b8986592814 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:50.434993 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6c6cc-7c44-4627-9e4e-00fb7e7f7cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:50.436839 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6c6cc-7c44-4627-9e4e-00fb7e7f7cd1 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:50.437301 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6c6cc-7c44-4627-9e4e-00fb7e7f7cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:50.437760 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6c6cc-7c44-4627-9e4e-00fb7e7f7cd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:50.495608 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 416/861] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:50 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 08:56:50.497871 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e7af0-4d91-4087-8d99-84f899b439b1 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:50.522686 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:50.528403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe6c6cc-7c44-4627-9e4e-00fb7e7f7cd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:50.529859 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe6c6cc-7c44-4627-9e4e-00fb7e7f7cd1 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:50.546920 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:50.602943 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e7af0-4d91-4087-8d99-84f899b439b1 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:50.633809 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 446/862] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:50 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 08:56:50.634971 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6c6cc-7c44-4627-9e4e-00fb7e7f7cd1 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:50.637269 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb268d15-dee2-4115-99f8-1e8757cb826e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:50.637896 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb268d15-dee2-4115-99f8-1e8757cb826e None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:50.638256 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb268d15-dee2-4115-99f8-1e8757cb826e None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:50.638579 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb268d15-dee2-4115-99f8-1e8757cb826e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:50.649515 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb268d15-dee2-4115-99f8-1e8757cb826e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:50.650939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb268d15-dee2-4115-99f8-1e8757cb826e None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:50.666787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-cb268d15-dee2-4115-99f8-1e8757cb826e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:50.666787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:50.666787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:50.666787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:50.666787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:50.666787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:50.666787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:50.666787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:50.666787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:50.666787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:50.666787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:50.666787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:50.666787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:50.666787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:50.666787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:50.666787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:50.667806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:50.667806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:50.667806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:50.667806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:50.667806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:50.667806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:50.667806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:50.667806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:50.667806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:50.667806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:50.667806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:50.667806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:50.667806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:50.667806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:50.667806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:50.667806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:50.667806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:50.673607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:50.673607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:50.673607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:50.673607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 417/863] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:50.673607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb268d15-dee2-4115-99f8-1e8757cb826e None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:50.745495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:50.751819 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6c6cc-7c44-4627-9e4e-00fb7e7f7cd1 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:50.768612 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:50.770546 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb268d15-dee2-4115-99f8-1e8757cb826e None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:50.788305 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08418c-804f-47e2-bdf6-53fe57c37d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:50.788910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08418c-804f-47e2-bdf6-53fe57c37d1d None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:50.789186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08418c-804f-47e2-bdf6-53fe57c37d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:50.789455 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08418c-804f-47e2-bdf6-53fe57c37d1d None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:50.790098 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08418c-804f-47e2-bdf6-53fe57c37d1d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:50.791488 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08418c-804f-47e2-bdf6-53fe57c37d1d None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:50.805467 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2306ffd-e852-41f7-9a41-28feeae1f023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:50.806090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2306ffd-e852-41f7-9a41-28feeae1f023 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:50.806367 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2306ffd-e852-41f7-9a41-28feeae1f023 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:50.806633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2306ffd-e852-41f7-9a41-28feeae1f023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:50.811120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 447/864] 10.222.0.36 () {60 vars in 1141 bytes} [Fri Sep 2 08:56:50 2022] POST /identity/v3/services => generated 269 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:56:50.811120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08418c-804f-47e2-bdf6-53fe57c37d1d None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:50.827768 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2306ffd-e852-41f7-9a41-28feeae1f023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:50.830686 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2306ffd-e852-41f7-9a41-28feeae1f023 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:50.853771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-c2306ffd-e852-41f7-9a41-28feeae1f023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:50.853771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:50.853771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:50.853771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:50.853771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:50.853771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:50.853771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:50.853771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:50.853771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:50.853771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:50.853771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:50.853771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:50.853771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:50.853771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:50.853771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:50.853771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:50.855027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:50.855027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:50.855027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:50.855027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:50.855027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:50.855027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:50.855027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:50.855027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:50.855027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:50.855027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:50.855027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:50.855027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:50.855027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:50.855027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:50.855027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:50.855027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:50.855027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:50.856179 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:50.856179 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:50.856179 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:50.858455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 418/865] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:50.864213 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2306ffd-e852-41f7-9a41-28feeae1f023 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:50.961244 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08418c-804f-47e2-bdf6-53fe57c37d1d None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:50.990983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2306ffd-e852-41f7-9a41-28feeae1f023 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:51.007104 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a9054-1dfa-4193-b35c-85f4c2c26633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:51.008363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a9054-1dfa-4193-b35c-85f4c2c26633 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:51.008363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a9054-1dfa-4193-b35c-85f4c2c26633 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:51.008363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a9054-1dfa-4193-b35c-85f4c2c26633 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:51.009130 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a9054-1dfa-4193-b35c-85f4c2c26633 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:51.009491 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a9054-1dfa-4193-b35c-85f4c2c26633 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:51.010918 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a9054-1dfa-4193-b35c-85f4c2c26633 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:51.026903 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 448/866] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:51.032634 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:51.032877 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a9054-1dfa-4193-b35c-85f4c2c26633 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:51.043702 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d68d91-2fc7-4361-ada5-5d8f135f1905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:51.044108 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d68d91-2fc7-4361-ada5-5d8f135f1905 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:51.044757 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d68d91-2fc7-4361-ada5-5d8f135f1905 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:51.045093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d68d91-2fc7-4361-ada5-5d8f135f1905 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:51.045858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d68d91-2fc7-4361-ada5-5d8f135f1905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:51.046229 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d68d91-2fc7-4361-ada5-5d8f135f1905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:51.047746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d68d91-2fc7-4361-ada5-5d8f135f1905 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:51.061201 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 419/867] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:51.066823 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d68d91-2fc7-4361-ada5-5d8f135f1905 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:51.073681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:51.171435 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a9054-1dfa-4193-b35c-85f4c2c26633 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:51.192168 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d68d91-2fc7-4361-ada5-5d8f135f1905 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:51.209697 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1493fea-f3dc-4dfe-8300-43dd35fbfc11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:51.210553 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1493fea-f3dc-4dfe-8300-43dd35fbfc11 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:51.210904 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1493fea-f3dc-4dfe-8300-43dd35fbfc11 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:51.211359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1493fea-f3dc-4dfe-8300-43dd35fbfc11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:51.218551 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1493fea-f3dc-4dfe-8300-43dd35fbfc11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:51.220053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1493fea-f3dc-4dfe-8300-43dd35fbfc11 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:51.230150 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d0164-c51f-4eee-b60f-a5a09cbc8cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:51.230974 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d0164-c51f-4eee-b60f-a5a09cbc8cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:51.231422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d0164-c51f-4eee-b60f-a5a09cbc8cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:51.231830 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6d0164-c51f-4eee-b60f-a5a09cbc8cf5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:51.232921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-c1493fea-f3dc-4dfe-8300-43dd35fbfc11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:51.232921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:51.232921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:51.232921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:51.232921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:51.232921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:51.232921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:51.232921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:51.232921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:51.232921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:51.232921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:51.232921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:51.232921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:51.232921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:51.232921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:51.233978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:51.233978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:51.233978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:51.233978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:51.233978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:51.233978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:51.233978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:51.233978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:51.233978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:51.233978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:51.233978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:51.233978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:51.233978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:51.233978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:51.233978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:51.233978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:51.233978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:51.235049 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:51.235049 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:51.235049 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:51.235049 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:51.235049 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:51.235049 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:51.235049 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:51.235049 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:51.238009 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 449/868] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:51.243885 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1493fea-f3dc-4dfe-8300-43dd35fbfc11 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:51.244269 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d0164-c51f-4eee-b60f-a5a09cbc8cf5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:51.245840 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6d0164-c51f-4eee-b60f-a5a09cbc8cf5 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:51.259613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-3a6d0164-c51f-4eee-b60f-a5a09cbc8cf5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:51.259613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:51.259613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:51.259613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:51.259613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:51.259613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:51.259613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:51.259613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:51.259613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:51.259613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:51.259613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:51.259613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:51.259613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:51.259613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:51.259613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:51.260733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:51.260733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:51.260733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:51.260733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:51.260733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:51.260733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:51.260733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:51.260733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:51.260733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:51.260733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:51.260733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:51.260733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:51.260733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:51.260733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:51.260733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:51.260733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:51.260733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:51.261811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:51.261811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:51.261811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:51.261811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:51.261811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:51.261811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:51.261811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:51.261811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:51.264177 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 420/869] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:51.269864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d0164-c51f-4eee-b60f-a5a09cbc8cf5 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:51.339183 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:51.379478 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:51.389963 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1493fea-f3dc-4dfe-8300-43dd35fbfc11 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:51.407193 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6d0164-c51f-4eee-b60f-a5a09cbc8cf5 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:51.432623 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595130c-5970-4687-958c-964317fffc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:51.433835 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595130c-5970-4687-958c-964317fffc57 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:51.434589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595130c-5970-4687-958c-964317fffc57 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:51.435477 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4595130c-5970-4687-958c-964317fffc57 None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:51.437013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4595130c-5970-4687-958c-964317fffc57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:51.437725 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4595130c-5970-4687-958c-964317fffc57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:51.440018 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4595130c-5970-4687-958c-964317fffc57 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:51.442897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854d64b-7f87-4bd7-9dd4-1cccd9fa5ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:51.443632 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854d64b-7f87-4bd7-9dd4-1cccd9fa5ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:51.443915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854d64b-7f87-4bd7-9dd4-1cccd9fa5ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:51.444186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854d64b-7f87-4bd7-9dd4-1cccd9fa5ca8 None admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:51.444927 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854d64b-7f87-4bd7-9dd4-1cccd9fa5ca8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:51.445233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854d64b-7f87-4bd7-9dd4-1cccd9fa5ca8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:51.446643 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1854d64b-7f87-4bd7-9dd4-1cccd9fa5ca8 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:51.456032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 450/870] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:51.462093 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595130c-5970-4687-958c-964317fffc57 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:51.464863 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 421/871] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:51.470621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854d64b-7f87-4bd7-9dd4-1cccd9fa5ca8 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:51.581151 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854d64b-7f87-4bd7-9dd4-1cccd9fa5ca8 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:51.588706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4595130c-5970-4687-958c-964317fffc57 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:51.594311 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:51.629014 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e4053-2689-467e-b8ef-f5e40830defa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:51.629700 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1de70-3c11-4014-88a3-5cd8f939be69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:51.629832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e4053-2689-467e-b8ef-f5e40830defa None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:51.630204 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e4053-2689-467e-b8ef-f5e40830defa None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:51.630571 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e4053-2689-467e-b8ef-f5e40830defa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:51.630862 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1de70-3c11-4014-88a3-5cd8f939be69 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:51.631314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1de70-3c11-4014-88a3-5cd8f939be69 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:51.631659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e4053-2689-467e-b8ef-f5e40830defa None admin] RBAC: Adding query filter params (role.id=637ed3cd3d484839ab21de02c2bbe293, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=76ebecce283f491e82a961d7f739878f) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:51.631865 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1de70-3c11-4014-88a3-5cd8f939be69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:51.632145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e4053-2689-467e-b8ef-f5e40830defa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:51.632892 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c1de70-3c11-4014-88a3-5cd8f939be69 None admin] RBAC: Adding query filter params (role.id=a0a76de72fba4036bc1e3e64df717c7f, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=92318948dcf1407b9fc8aa5c199787ae) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:51.633350 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c1de70-3c11-4014-88a3-5cd8f939be69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:51.633851 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e4053-2689-467e-b8ef-f5e40830defa None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:51.635045 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c1de70-3c11-4014-88a3-5cd8f939be69 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:51.642503 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 422/872] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:56:51 2022] GET /identity/v3/role_assignments?user.id=92318948dcf1407b9fc8aa5c199787ae&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=a0a76de72fba4036bc1e3e64df717c7f => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:51.647565 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 451/873] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:56:51 2022] GET /identity/v3/role_assignments?user.id=76ebecce283f491e82a961d7f739878f&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=637ed3cd3d484839ab21de02c2bbe293 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:52.100884 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46457e2a-b5ef-429b-92e2-c5a95edeecd1 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:52.101273 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46457e2a-b5ef-429b-92e2-c5a95edeecd1 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:52.101455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46457e2a-b5ef-429b-92e2-c5a95edeecd1 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:52.102993 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 423/874] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:52.108240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9e763-fe56-4442-9c00-d3627ca625b4 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:52.108522 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9e763-fe56-4442-9c00-d3627ca625b4 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:52.108798 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9e763-fe56-4442-9c00-d3627ca625b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:52.128068 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-1cf9e763-fe56-4442-9c00-d3627ca625b4 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:52.129231 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-1cf9e763-fe56-4442-9c00-d3627ca625b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:52.140667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:52.201518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 452/875] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:52 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:56:52.469704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46457e2a-b5ef-429b-92e2-c5a95edeecd1 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:52.573121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46457e2a-b5ef-429b-92e2-c5a95edeecd1 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:52.607520 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e08f5-23f7-4f78-b113-b5d12d615907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:52.608133 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e08f5-23f7-4f78-b113-b5d12d615907 None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:52.608380 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e08f5-23f7-4f78-b113-b5d12d615907 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:52.608571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e08f5-23f7-4f78-b113-b5d12d615907 None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:52.609223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e08f5-23f7-4f78-b113-b5d12d615907 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:52.610564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e08f5-23f7-4f78-b113-b5d12d615907 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:52.629123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 424/876] 10.222.0.36 () {60 vars in 1141 bytes} [Fri Sep 2 08:56:52 2022] POST /identity/v3/services => generated 251 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:56:52.820751 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e899b-1829-4cdc-b149-f87554d4c100 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:52.821286 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e899b-1829-4cdc-b149-f87554d4c100 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:52.821286 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e899b-1829-4cdc-b149-f87554d4c100 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:52.822992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 453/877] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:52.828735 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af216d7c-1a48-4836-a5bf-176b07b19352 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:52.829046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af216d7c-1a48-4836-a5bf-176b07b19352 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:52.829291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af216d7c-1a48-4836-a5bf-176b07b19352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:52.848661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-af216d7c-1a48-4836-a5bf-176b07b19352 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:52.849876 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-af216d7c-1a48-4836-a5bf-176b07b19352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:52.878350 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:52.930015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 425/878] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:52 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:56:53.201528 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e899b-1829-4cdc-b149-f87554d4c100 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:53.311587 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e899b-1829-4cdc-b149-f87554d4c100 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:53.349707 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8da71c-989d-488d-81d3-287d61f6dad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:53.350330 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8da71c-989d-488d-81d3-287d61f6dad0 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:53.350608 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8da71c-989d-488d-81d3-287d61f6dad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:53.350874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8da71c-989d-488d-81d3-287d61f6dad0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:53.351541 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8da71c-989d-488d-81d3-287d61f6dad0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:53.352972 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8da71c-989d-488d-81d3-287d61f6dad0 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:53.362691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-af8da71c-989d-488d-81d3-287d61f6dad0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 08:56:53.362691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:53.362691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:56:53.362691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:56:53.362691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:56:53.362691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:56:53.362691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:56:53.362691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:53.362691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 08:56:53.362691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:53.362691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:56:53.362691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:53.362691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:53.362691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:53.362691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:53.362691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:53.363713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:53.363713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:53.363713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:53.363713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:53.363713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:53.363713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:53.363713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:53.363713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:56:53.363713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:56:53.363713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:56:53.363713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:56:53.363713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:53.363713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:53.363713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:53.363713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:53.363713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:53.363713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:53.364615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:53.364615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:53.364615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:53.364615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:53.364615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:53.364615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:53.364615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:53.364615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:53.364615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:53.364615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:53.364615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:56:53.364615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:53.364615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 08:56:53.364615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:53.364615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 454/879] 10.222.0.36 () {56 vars in 1132 bytes} [Fri Sep 2 08:56:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:53.370041 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af216d7c-1a48-4836-a5bf-176b07b19352 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:53.470046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af216d7c-1a48-4836-a5bf-176b07b19352 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:53.505198 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b81ddb-1c70-4c15-9415-1eb31f48c6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:53.505800 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b81ddb-1c70-4c15-9415-1eb31f48c6d8 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:53.506072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b81ddb-1c70-4c15-9415-1eb31f48c6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:53.506333 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b81ddb-1c70-4c15-9415-1eb31f48c6d8 None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:53.507015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b81ddb-1c70-4c15-9415-1eb31f48c6d8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:53.507351 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b81ddb-1c70-4c15-9415-1eb31f48c6d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:53.508739 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b81ddb-1c70-4c15-9415-1eb31f48c6d8 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:53.513774 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 426/880] 10.222.0.36 () {56 vars in 1126 bytes} [Fri Sep 2 08:56:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:53.519053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8da71c-989d-488d-81d3-287d61f6dad0 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:53.547433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:53.633914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8da71c-989d-488d-81d3-287d61f6dad0 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:53.669111 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df0b80-e1e0-48c1-912b-a46bda8f3d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:53.669717 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df0b80-e1e0-48c1-912b-a46bda8f3d8f None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:53.669995 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df0b80-e1e0-48c1-912b-a46bda8f3d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:53.670263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df0b80-e1e0-48c1-912b-a46bda8f3d8f None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:53.670885 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df0b80-e1e0-48c1-912b-a46bda8f3d8f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:53.671211 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df0b80-e1e0-48c1-912b-a46bda8f3d8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:53.672629 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df0b80-e1e0-48c1-912b-a46bda8f3d8f None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:53.678576 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 455/881] 10.222.0.36 () {56 vars in 1126 bytes} [Fri Sep 2 08:56:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:53.684378 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b81ddb-1c70-4c15-9415-1eb31f48c6d8 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:53.710052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:53.723531 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8614aed-d9cd-41c3-818e-75fa96aee04c None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:53.723731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8614aed-d9cd-41c3-818e-75fa96aee04c None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:53.723988 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8614aed-d9cd-41c3-818e-75fa96aee04c None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:53.725726 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 456/882] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:53.732324 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47931175-a79f-47da-84c6-066eed69b2b4 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:53.732608 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47931175-a79f-47da-84c6-066eed69b2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:53.732890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47931175-a79f-47da-84c6-066eed69b2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:53.756653 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-47931175-a79f-47da-84c6-066eed69b2b4 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:53.758456 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-47931175-a79f-47da-84c6-066eed69b2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:53.800734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b81ddb-1c70-4c15-9415-1eb31f48c6d8 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:53.835406 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea9e60e-5564-4b33-bc0d-bb4f87ee80fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:53.836006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9e60e-5564-4b33-bc0d-bb4f87ee80fa None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:53.836287 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9e60e-5564-4b33-bc0d-bb4f87ee80fa None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:53.836566 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9e60e-5564-4b33-bc0d-bb4f87ee80fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:53.837255 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9e60e-5564-4b33-bc0d-bb4f87ee80fa None admin] RBAC: Adding query filter params (interface=public, service_id=021361b6eaf4490698ccbb7bebf227bf, region_id=RegionOne) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:53.837562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9e60e-5564-4b33-bc0d-bb4f87ee80fa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:53.838912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea9e60e-5564-4b33-bc0d-bb4f87ee80fa None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:53.843802 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 427/883] 10.222.0.36 () {56 vars in 1251 bytes} [Fri Sep 2 08:56:53 2022] GET /identity/v3/endpoints?service_id=021361b6eaf4490698ccbb7bebf227bf&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:53.851103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6082e-90db-4346-9e7b-9969d3456684 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:53.851103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6082e-90db-4346-9e7b-9969d3456684 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:53.851103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6082e-90db-4346-9e7b-9969d3456684 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:53.851385 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 457/884] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:53 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:56:53.852125 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47931175-a79f-47da-84c6-066eed69b2b4 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:53.855593 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 428/885] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:53 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:53.856899 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc37f2-b463-4e6c-86f5-878094e80e49 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:53.857185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc37f2-b463-4e6c-86f5-878094e80e49 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:53.857455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc37f2-b463-4e6c-86f5-878094e80e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:53.867240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:53.883427 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-bfbc37f2-b463-4e6c-86f5-878094e80e49 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:53.885158 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-bfbc37f2-b463-4e6c-86f5-878094e80e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:53.966692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47931175-a79f-47da-84c6-066eed69b2b4 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:53.972568 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 429/886] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:53 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 08:56:54.009365 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3ea28-2013-4755-8f84-6e711100c10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:54.009900 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3ea28-2013-4755-8f84-6e711100c10d None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:54.010182 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3ea28-2013-4755-8f84-6e711100c10d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:54.010458 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3ea28-2013-4755-8f84-6e711100c10d None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:54.011061 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3ea28-2013-4755-8f84-6e711100c10d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:54.012515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3ea28-2013-4755-8f84-6e711100c10d None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:54.018803 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 458/887] 10.222.0.36 () {56 vars in 1087 bytes} [Fri Sep 2 08:56:53 2022] GET /identity/v3/services => generated 1268 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 08:56:54.136820 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc37f2-b463-4e6c-86f5-878094e80e49 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:54.156385 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:54.241907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc37f2-b463-4e6c-86f5-878094e80e49 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:54.244170 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3ea28-2013-4755-8f84-6e711100c10d None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:54.281346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae2c4b3-5e0b-4bfa-80fe-adfe1d974dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:54.281912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2c4b3-5e0b-4bfa-80fe-adfe1d974dec None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:54.282194 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2c4b3-5e0b-4bfa-80fe-adfe1d974dec None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:54.282470 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2c4b3-5e0b-4bfa-80fe-adfe1d974dec None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:54.293630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae2c4b3-5e0b-4bfa-80fe-adfe1d974dec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:54.299562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-8ae2c4b3-5e0b-4bfa-80fe-adfe1d974dec None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 08:56:54.299562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:54.299562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:54.299562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:54.299562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:54.299562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:54.299562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:54.299562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:54.299562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:54.299562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:54.299562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:54.299562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:54.299562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:54.299562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:54.299562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:54.299562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:54.300617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:54.300617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:54.300617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:54.300617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:54.300617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:54.300617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:54.300617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:54.300617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:54.300617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:54.300617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:54.300617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:54.300617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:54.300617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:54.300617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:54.300617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:54.300617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:54.302004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:54.302004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:54.302004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:54.302004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:54.302004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:54.302004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:54.302004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:54.302004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:54.302004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:54.302004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:54.302004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 08:56:54.302004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:54.302004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 430/888] 10.222.0.36 () {56 vars in 1120 bytes} [Fri Sep 2 08:56:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:54.308017 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae2c4b3-5e0b-4bfa-80fe-adfe1d974dec None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:54.327179 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:54.375432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3ea28-2013-4755-8f84-6e711100c10d None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:54.412809 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19220394-30df-4f31-9214-c5e5c157af7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:54.413815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19220394-30df-4f31-9214-c5e5c157af7e None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:54.414304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19220394-30df-4f31-9214-c5e5c157af7e None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:54.414771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19220394-30df-4f31-9214-c5e5c157af7e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:54.415585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19220394-30df-4f31-9214-c5e5c157af7e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:54.417161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19220394-30df-4f31-9214-c5e5c157af7e None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:54.424434 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:54.432276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-19220394-30df-4f31-9214-c5e5c157af7e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 08:56:54.432276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:54.432276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:56:54.432276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:56:54.432276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:56:54.432276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:56:54.432276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:56:54.432276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:54.432276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 08:56:54.432276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:54.432276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:56:54.432276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:54.432276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:54.432276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:54.432276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:54.432276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:54.432276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:54.433281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:54.433281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:54.433281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:54.433281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:54.433281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:54.433281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:54.433281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:56:54.433281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:56:54.433281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:56:54.433281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:56:54.433281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:54.433281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:54.433281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:54.433281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:54.433281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:54.433281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:54.434391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:54.434391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:54.434391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:54.434391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:54.434391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:54.434391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:54.434391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:54.434391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:54.434391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:54.434391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:54.434391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:56:54.434391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:54.434391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 08:56:54.434391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:54.434391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 459/889] 10.222.0.36 () {56 vars in 1126 bytes} [Fri Sep 2 08:56:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:54.440573 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19220394-30df-4f31-9214-c5e5c157af7e None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:54.456647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae2c4b3-5e0b-4bfa-80fe-adfe1d974dec None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:54.493958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec350e-198f-4acb-9c2e-cb3fdec86f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:54.494988 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec350e-198f-4acb-9c2e-cb3fdec86f3f None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:54.495463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec350e-198f-4acb-9c2e-cb3fdec86f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:54.495878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec350e-198f-4acb-9c2e-cb3fdec86f3f None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:54.496759 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec350e-198f-4acb-9c2e-cb3fdec86f3f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:54.497279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec350e-198f-4acb-9c2e-cb3fdec86f3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:54.498910 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec350e-198f-4acb-9c2e-cb3fdec86f3f None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:54.508365 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 431/890] 10.222.0.36 () {56 vars in 1115 bytes} [Fri Sep 2 08:56:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:54.514101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec350e-198f-4acb-9c2e-cb3fdec86f3f None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:54.563194 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19220394-30df-4f31-9214-c5e5c157af7e None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:54.595225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:54.602066 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc05a9a-f502-49bb-bee3-834a790368d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:54.603030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc05a9a-f502-49bb-bee3-834a790368d8 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:54.603572 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc05a9a-f502-49bb-bee3-834a790368d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:54.604053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc05a9a-f502-49bb-bee3-834a790368d8 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:54.604986 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc05a9a-f502-49bb-bee3-834a790368d8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:54.605497 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc05a9a-f502-49bb-bee3-834a790368d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:54.607175 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc05a9a-f502-49bb-bee3-834a790368d8 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:54.617606 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 460/891] 10.222.0.36 () {56 vars in 1122 bytes} [Fri Sep 2 08:56:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:54.618017 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b436af7-59f6-474b-abb3-a20ee28d71eb None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:54.618194 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b436af7-59f6-474b-abb3-a20ee28d71eb None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:54.618439 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b436af7-59f6-474b-abb3-a20ee28d71eb None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:54.620119 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 461/892] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:54.624583 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b436af7-59f6-474b-abb3-a20ee28d71eb None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:54.627379 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec350e-198f-4acb-9c2e-cb3fdec86f3f None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:54.664618 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ebd71-e0d2-48c1-a690-29704115139a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:54.665245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ebd71-e0d2-48c1-a690-29704115139a None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:54.665937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ebd71-e0d2-48c1-a690-29704115139a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:54.666216 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ebd71-e0d2-48c1-a690-29704115139a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:54.674830 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:54.678785 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553ebd71-e0d2-48c1-a690-29704115139a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:54.680986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553ebd71-e0d2-48c1-a690-29704115139a None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:54.697525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-553ebd71-e0d2-48c1-a690-29704115139a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:54.697525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:54.697525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:54.697525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:54.697525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:54.697525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:54.697525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:54.697525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:54.697525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:54.697525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:54.697525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:54.697525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:54.697525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:54.697525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:54.697525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:54.697525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:54.698948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:54.698948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:54.698948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:54.698948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:54.698948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:54.698948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:54.698948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:54.698948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:54.698948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:54.698948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:54.698948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:54.698948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:54.698948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:54.698948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:54.698948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:54.698948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:54.698948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:54.699914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:54.699914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:54.699914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:54.703599 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 432/893] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:54.704824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7c723-f984-47e7-8d35-3638682f224e None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:54.705146 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7c723-f984-47e7-8d35-3638682f224e None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:54.705427 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7c723-f984-47e7-8d35-3638682f224e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:54.726074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-f6d7c723-f984-47e7-8d35-3638682f224e None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:54.727547 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-f6d7c723-f984-47e7-8d35-3638682f224e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:54.741060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b436af7-59f6-474b-abb3-a20ee28d71eb None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:54.775832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41311317-8ee3-4dc5-bbc6-1231aa1e0d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:54.776717 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41311317-8ee3-4dc5-bbc6-1231aa1e0d4d None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:54.776717 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41311317-8ee3-4dc5-bbc6-1231aa1e0d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:54.776954 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41311317-8ee3-4dc5-bbc6-1231aa1e0d4d None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:54.777573 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41311317-8ee3-4dc5-bbc6-1231aa1e0d4d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:54.777871 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41311317-8ee3-4dc5-bbc6-1231aa1e0d4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:54.779288 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41311317-8ee3-4dc5-bbc6-1231aa1e0d4d None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:54.791100 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 462/894] 10.222.0.36 () {56 vars in 1122 bytes} [Fri Sep 2 08:56:54 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:54.791100 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41311317-8ee3-4dc5-bbc6-1231aa1e0d4d None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:54.812705 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 433/895] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:56:54.857046 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:54.903571 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41311317-8ee3-4dc5-bbc6-1231aa1e0d4d None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:54.941556 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e7ba5-0d47-4b20-8636-11deac91c517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:54.942165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e7ba5-0d47-4b20-8636-11deac91c517 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:54.942444 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e7ba5-0d47-4b20-8636-11deac91c517 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:54.942717 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e7ba5-0d47-4b20-8636-11deac91c517 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:54.943389 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e7ba5-0d47-4b20-8636-11deac91c517 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:54.943693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e7ba5-0d47-4b20-8636-11deac91c517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:54.945060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e7ba5-0d47-4b20-8636-11deac91c517 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:54.957429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 463/896] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:54.962866 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7c723-f984-47e7-8d35-3638682f224e None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:55.074793 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7c723-f984-47e7-8d35-3638682f224e None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:55.083986 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e7ba5-0d47-4b20-8636-11deac91c517 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:55.103680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:55.124543 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c792bc-8b3d-4117-b4f0-deb7a287e5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:55.125625 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c792bc-8b3d-4117-b4f0-deb7a287e5bf None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:55.126087 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c792bc-8b3d-4117-b4f0-deb7a287e5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:55.126554 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c792bc-8b3d-4117-b4f0-deb7a287e5bf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:55.138068 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c792bc-8b3d-4117-b4f0-deb7a287e5bf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:55.140645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c792bc-8b3d-4117-b4f0-deb7a287e5bf None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:55.158704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-81c792bc-8b3d-4117-b4f0-deb7a287e5bf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 08:56:55.158704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:55.158704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:55.158704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:55.158704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:55.158704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:55.158704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:55.158704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:55.158704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:55.158704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:55.158704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:55.158704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:55.158704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:55.158704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:55.158704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:55.158704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:55.159780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:55.159780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:55.159780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:55.159780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:55.159780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:55.159780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:55.159780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:55.159780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:55.159780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:55.159780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:55.159780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:55.159780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:55.159780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:55.159780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:55.159780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:55.159780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:55.159780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:55.159780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:55.160827 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:55.160827 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:55.160827 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:55.160827 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:55.160827 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:55.160827 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:55.160827 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:55.160827 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:55.160827 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 08:56:55.160827 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:55.163810 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 434/897] 10.222.0.36 () {56 vars in 1117 bytes} [Fri Sep 2 08:56:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:55.170114 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c792bc-8b3d-4117-b4f0-deb7a287e5bf None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:55.211493 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e7ba5-0d47-4b20-8636-11deac91c517 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:55.234995 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:55.250996 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff639034-91c1-4fef-8c8f-43d31070ac0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:55.251661 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff639034-91c1-4fef-8c8f-43d31070ac0e None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:55.251969 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff639034-91c1-4fef-8c8f-43d31070ac0e None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:55.252237 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff639034-91c1-4fef-8c8f-43d31070ac0e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:55.252815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff639034-91c1-4fef-8c8f-43d31070ac0e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:55.254282 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff639034-91c1-4fef-8c8f-43d31070ac0e None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:55.263804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-ff639034-91c1-4fef-8c8f-43d31070ac0e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 08:56:55.263804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:55.263804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:56:55.263804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:56:55.263804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:56:55.263804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:56:55.263804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:56:55.263804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:55.263804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 08:56:55.263804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:55.263804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:56:55.263804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:55.263804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:55.263804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:55.263804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:55.263804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:55.263804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:55.264817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:55.264817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:55.264817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:55.264817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:55.264817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:55.264817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:55.264817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:56:55.264817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:56:55.264817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:56:55.264817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:56:55.264817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:55.264817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:55.264817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:55.264817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:55.264817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:55.264817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:55.265676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:55.265676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:55.265676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:55.265676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:55.265676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:55.265676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:55.265676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:55.265676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:55.265676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:55.265676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:55.265676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:56:55.265676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:55.265676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 08:56:55.265676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:55.265676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 464/898] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:56:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:55.271749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff639034-91c1-4fef-8c8f-43d31070ac0e None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:55.302599 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c792bc-8b3d-4117-b4f0-deb7a287e5bf None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:55.344973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc0142-45a1-4785-b521-6249b98c13f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:55.345659 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc0142-45a1-4785-b521-6249b98c13f2 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:55.346431 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc0142-45a1-4785-b521-6249b98c13f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:55.346694 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc0142-45a1-4785-b521-6249b98c13f2 None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:55.347462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc0142-45a1-4785-b521-6249b98c13f2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:55.347764 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc0142-45a1-4785-b521-6249b98c13f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:55.349197 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc0142-45a1-4785-b521-6249b98c13f2 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:55.371421 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 435/899] 10.222.0.36 () {56 vars in 1149 bytes} [Fri Sep 2 08:56:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:55.372316 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc0142-45a1-4785-b521-6249b98c13f2 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:55.396738 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff639034-91c1-4fef-8c8f-43d31070ac0e None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:55.413991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:55.442396 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4375bd-b6c9-4f2f-a611-ef93112da498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:55.443273 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4375bd-b6c9-4f2f-a611-ef93112da498 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:55.443623 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4375bd-b6c9-4f2f-a611-ef93112da498 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:55.443963 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4375bd-b6c9-4f2f-a611-ef93112da498 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:55.445145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4375bd-b6c9-4f2f-a611-ef93112da498 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:55.445529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4375bd-b6c9-4f2f-a611-ef93112da498 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:55.447285 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4375bd-b6c9-4f2f-a611-ef93112da498 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:55.453564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 465/900] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:56:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:55.459347 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4375bd-b6c9-4f2f-a611-ef93112da498 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:55.482589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc0142-45a1-4785-b521-6249b98c13f2 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:55.520820 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26982be7-b51c-4377-91da-373e67ee7f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:55.521620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26982be7-b51c-4377-91da-373e67ee7f2c None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:55.521989 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26982be7-b51c-4377-91da-373e67ee7f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:55.522368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26982be7-b51c-4377-91da-373e67ee7f2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:55.524596 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:55.579791 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4375bd-b6c9-4f2f-a611-ef93112da498 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:55.594496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26982be7-b51c-4377-91da-373e67ee7f2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:55.596239 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26982be7-b51c-4377-91da-373e67ee7f2c None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:55.616268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7fcf1c-960f-4dfe-b08f-13c69ea39d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:55.617176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fcf1c-960f-4dfe-b08f-13c69ea39d36 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:55.617632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fcf1c-960f-4dfe-b08f-13c69ea39d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:55.618086 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fcf1c-960f-4dfe-b08f-13c69ea39d36 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:55.618898 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7fcf1c-960f-4dfe-b08f-13c69ea39d36 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:55.619452 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7fcf1c-960f-4dfe-b08f-13c69ea39d36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:55.621017 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7fcf1c-960f-4dfe-b08f-13c69ea39d36 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:55.626557 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 466/901] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:56:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:55.632605 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7fcf1c-960f-4dfe-b08f-13c69ea39d36 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:55.643947 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:55.714338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 436/902] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:56:55 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 08:56:55.716001 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26982be7-b51c-4377-91da-373e67ee7f2c None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:55.741212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7fcf1c-960f-4dfe-b08f-13c69ea39d36 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:55.755142 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:55.779462 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8cf29-720d-4018-b5f3-2e0cac341f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:55.780077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8cf29-720d-4018-b5f3-2e0cac341f44 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:55.780359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8cf29-720d-4018-b5f3-2e0cac341f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:55.780648 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8cf29-720d-4018-b5f3-2e0cac341f44 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:55.781336 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8cf29-720d-4018-b5f3-2e0cac341f44 None admin] RBAC: Adding query filter params (interface=public, service_id=310a10f3a7614cf487c27cacb86e2fc3, region_id=RegionOne) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:55.781645 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8cf29-720d-4018-b5f3-2e0cac341f44 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:55.783053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8cf29-720d-4018-b5f3-2e0cac341f44 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:55.795901 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 467/903] 10.222.0.36 () {56 vars in 1251 bytes} [Fri Sep 2 08:56:55 2022] GET /identity/v3/endpoints?service_id=310a10f3a7614cf487c27cacb86e2fc3&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:55.796797 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8cf29-720d-4018-b5f3-2e0cac341f44 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:55.825989 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26982be7-b51c-4377-91da-373e67ee7f2c None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:55.862186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805812f-f3f3-47d8-b5f1-9195fc043b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:55.863192 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805812f-f3f3-47d8-b5f1-9195fc043b8a None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:55.863695 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805812f-f3f3-47d8-b5f1-9195fc043b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:55.864155 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805812f-f3f3-47d8-b5f1-9195fc043b8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:55.871052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:55.875844 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805812f-f3f3-47d8-b5f1-9195fc043b8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:55.878204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805812f-f3f3-47d8-b5f1-9195fc043b8a None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:55.894327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-b805812f-f3f3-47d8-b5f1-9195fc043b8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:55.894327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:55.894327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:55.894327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:55.894327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:55.894327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:55.894327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:55.894327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:55.894327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:55.894327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:55.894327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:55.894327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:55.894327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:55.894327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:55.894327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:55.894327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:55.895381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:55.895381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:55.895381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:55.895381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:55.895381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:55.895381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:55.895381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:55.895381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:55.895381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:55.895381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:55.895381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:55.895381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:55.895381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:55.895381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:55.895381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:55.895381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:55.895381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:55.896285 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:55.896285 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:55.896285 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:55.898567 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 437/904] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:55.904187 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805812f-f3f3-47d8-b5f1-9195fc043b8a None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:55.906627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8cf29-720d-4018-b5f3-2e0cac341f44 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:55.945002 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe3d35-700f-4c29-b880-ab965a5fd7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:55.945646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe3d35-700f-4c29-b880-ab965a5fd7ae None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:55.945977 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe3d35-700f-4c29-b880-ab965a5fd7ae None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:55.946251 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe3d35-700f-4c29-b880-ab965a5fd7ae None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:55.946860 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe3d35-700f-4c29-b880-ab965a5fd7ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:55.948378 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe3d35-700f-4c29-b880-ab965a5fd7ae None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:55.960642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 468/905] 10.222.0.36 () {56 vars in 1087 bytes} [Fri Sep 2 08:56:55 2022] GET /identity/v3/services => generated 1268 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 08:56:55.987061 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699aaa68-d38c-47ca-8c8c-1d369bf00ec9 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:55.987372 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699aaa68-d38c-47ca-8c8c-1d369bf00ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:55.987591 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699aaa68-d38c-47ca-8c8c-1d369bf00ec9 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:55.989210 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 469/906] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:56:55.995101 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d574559-8e7d-43e0-a6f6-7a119c75ef56 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:55.995596 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d574559-8e7d-43e0-a6f6-7a119c75ef56 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:55.996014 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d574559-8e7d-43e0-a6f6-7a119c75ef56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:56.018447 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-0d574559-8e7d-43e0-a6f6-7a119c75ef56 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:56.020304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0d574559-8e7d-43e0-a6f6-7a119c75ef56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:56.020971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805812f-f3f3-47d8-b5f1-9195fc043b8a None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:56.059679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4aca4-6d20-44dd-a66e-982c907388d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:56.060343 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4aca4-6d20-44dd-a66e-982c907388d1 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:56.060733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4aca4-6d20-44dd-a66e-982c907388d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:56.061050 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4aca4-6d20-44dd-a66e-982c907388d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:56.061708 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4aca4-6d20-44dd-a66e-982c907388d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:56.062055 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4aca4-6d20-44dd-a66e-982c907388d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:56.063516 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4aca4-6d20-44dd-a66e-982c907388d1 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:56.074160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:56.083450 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 438/907] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:56.086726 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4aca4-6d20-44dd-a66e-982c907388d1 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:56.120746 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 470/908] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:55 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:56:56.158470 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:56.212792 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4aca4-6d20-44dd-a66e-982c907388d1 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:56.248845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109543f6-31c4-4e32-ac58-85df6e06d337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:56.249393 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109543f6-31c4-4e32-ac58-85df6e06d337 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:56.249668 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109543f6-31c4-4e32-ac58-85df6e06d337 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:56.249934 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109543f6-31c4-4e32-ac58-85df6e06d337 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:56.256865 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109543f6-31c4-4e32-ac58-85df6e06d337 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:56.258391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109543f6-31c4-4e32-ac58-85df6e06d337 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:56.269938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-109543f6-31c4-4e32-ac58-85df6e06d337 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:56.269938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:56.269938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:56.269938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:56.269938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:56.269938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:56.269938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:56.269938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:56.269938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:56.269938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:56.269938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:56.269938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:56.269938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:56:56.269938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:56:56.269938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:56:56.270991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:56:56.270991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:56.270991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:56.270991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:56.270991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:56.270991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:56.270991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:56.270991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:56.270991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:56.270991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:56.270991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:56.270991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:56.270991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:56.270991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:56.270991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:56.270991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:56.270991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:56.271939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:56:56.271939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:56:56.271939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:56:56.271939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:56:56.271939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:56:56.271939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:56:56.271939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:56:56.271939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:56.275467 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 439/909] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:56.280212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d574559-8e7d-43e0-a6f6-7a119c75ef56 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:56.391378 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109543f6-31c4-4e32-ac58-85df6e06d337 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:56.393888 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d574559-8e7d-43e0-a6f6-7a119c75ef56 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:56.441688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30ecba-f45d-4b7d-8085-78882e5e987b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:56.442999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30ecba-f45d-4b7d-8085-78882e5e987b None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:56.443694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30ecba-f45d-4b7d-8085-78882e5e987b None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:56.444122 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb30ecba-f45d-4b7d-8085-78882e5e987b None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:56.445235 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30ecba-f45d-4b7d-8085-78882e5e987b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:56.446104 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30ecba-f45d-4b7d-8085-78882e5e987b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:56.447728 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb30ecba-f45d-4b7d-8085-78882e5e987b None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:56.464830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 471/910] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:56:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:56.472345 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30ecba-f45d-4b7d-8085-78882e5e987b None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:56.505115 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109543f6-31c4-4e32-ac58-85df6e06d337 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:56.512642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:56.518504 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:56.545977 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8cf58f-8e34-4337-8dbf-786c8569d89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:56.546983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8cf58f-8e34-4337-8dbf-786c8569d89b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:56.547544 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8cf58f-8e34-4337-8dbf-786c8569d89b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:56.548070 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8cf58f-8e34-4337-8dbf-786c8569d89b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:56.549521 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8cf58f-8e34-4337-8dbf-786c8569d89b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:56.551257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8cf58f-8e34-4337-8dbf-786c8569d89b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:56.561209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-db8cf58f-8e34-4337-8dbf-786c8569d89b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 08:56:56.561209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:56.561209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:56:56.561209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:56:56.561209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:56:56.561209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:56:56.561209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:56:56.561209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:56.561209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 08:56:56.561209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:56.561209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:56:56.561209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:56.561209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:56.561209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:56.561209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:56.561209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:56.562185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:56.562185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:56.562185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:56.562185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:56.562185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:56.562185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:56.562185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:56.562185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:56:56.562185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:56:56.562185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:56:56.562185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:56:56.562185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:56.562185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:56.562185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:56.562185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:56.562185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:56.562185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:56.563098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:56.563098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:56.563098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:56.563098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:56.563098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:56.563098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:56.563098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:56.563098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:56.563098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:56.563098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:56.563098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:56:56.563098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:56.563098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 08:56:56.563098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:56.570673 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 440/911] 10.222.0.36 () {56 vars in 1132 bytes} [Fri Sep 2 08:56:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:56.572318 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8cf58f-8e34-4337-8dbf-786c8569d89b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:56.603350 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb30ecba-f45d-4b7d-8085-78882e5e987b None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:56.649245 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74101826-4b39-4621-8971-099504f5017f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:56.649859 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74101826-4b39-4621-8971-099504f5017f None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:56.651007 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74101826-4b39-4621-8971-099504f5017f None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:56.651352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74101826-4b39-4621-8971-099504f5017f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:56.652094 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74101826-4b39-4621-8971-099504f5017f None admin] RBAC: Adding query filter params (role.id=a604599bb8af4abfaada3a92ac76b1bd, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=76ebecce283f491e82a961d7f739878f) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:56.652409 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74101826-4b39-4621-8971-099504f5017f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:56.653854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74101826-4b39-4621-8971-099504f5017f None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:56.667138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 472/912] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:56:56 2022] GET /identity/v3/role_assignments?user.id=76ebecce283f491e82a961d7f739878f&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=a604599bb8af4abfaada3a92ac76b1bd => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:56.688717 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8cf58f-8e34-4337-8dbf-786c8569d89b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:56.728976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca40fa-1655-4584-a485-d6def43e99e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:56.729615 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca40fa-1655-4584-a485-d6def43e99e7 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:56.729920 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca40fa-1655-4584-a485-d6def43e99e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:56.730121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca40fa-1655-4584-a485-d6def43e99e7 None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:56.730767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca40fa-1655-4584-a485-d6def43e99e7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:56.731087 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca40fa-1655-4584-a485-d6def43e99e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:56.732488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca40fa-1655-4584-a485-d6def43e99e7 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:56.738346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 441/913] 10.222.0.36 () {56 vars in 1126 bytes} [Fri Sep 2 08:56:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:56.748278 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74101826-4b39-4621-8971-099504f5017f None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:56.753894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ae4a5-c0c3-427f-8ac3-b4e36e99123b None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:56.754170 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ae4a5-c0c3-427f-8ac3-b4e36e99123b None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:56.754450 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ae4a5-c0c3-427f-8ac3-b4e36e99123b None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:56.756171 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 442/914] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:56:56.764624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5cd5c-363f-4a45-a12e-b945bac4b79d None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:56.764904 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5cd5c-363f-4a45-a12e-b945bac4b79d None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:56.765172 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b5cd5c-363f-4a45-a12e-b945bac4b79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:56.785881 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-64b5cd5c-363f-4a45-a12e-b945bac4b79d None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:56.787119 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-64b5cd5c-363f-4a45-a12e-b945bac4b79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:56.821953 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:56.865383 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 443/915] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:56:56.881785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:56.889283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74101826-4b39-4621-8971-099504f5017f None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:56.926414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3580fb7-23e5-43e3-8461-484456ae6ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:56.927090 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3580fb7-23e5-43e3-8461-484456ae6ae4 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:56.927336 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3580fb7-23e5-43e3-8461-484456ae6ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:56.927566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3580fb7-23e5-43e3-8461-484456ae6ae4 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:56.928188 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3580fb7-23e5-43e3-8461-484456ae6ae4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:56.928484 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3580fb7-23e5-43e3-8461-484456ae6ae4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:56.929960 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3580fb7-23e5-43e3-8461-484456ae6ae4 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:56.939740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 473/916] 10.222.0.36 () {56 vars in 1126 bytes} [Fri Sep 2 08:56:56 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:56.941905 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5cd5c-363f-4a45-a12e-b945bac4b79d None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:57.045183 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b5cd5c-363f-4a45-a12e-b945bac4b79d None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:57.083183 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ffa6d1-14b9-4a0c-bc28-3753c82982b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:57.083798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffa6d1-14b9-4a0c-bc28-3753c82982b2 None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:57.084078 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffa6d1-14b9-4a0c-bc28-3753c82982b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:57.084346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ffa6d1-14b9-4a0c-bc28-3753c82982b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:57.085001 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ffa6d1-14b9-4a0c-bc28-3753c82982b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:57.086349 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ffa6d1-14b9-4a0c-bc28-3753c82982b2 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:57.110487 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 444/917] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 08:56:56 2022] POST /identity/v3/endpoints => generated 347 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:56:57.145184 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3580fb7-23e5-43e3-8461-484456ae6ae4 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:57.264176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3580fb7-23e5-43e3-8461-484456ae6ae4 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:57.309466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ea3cb-8ea5-4295-a25f-547841d9fedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:57.309997 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ea3cb-8ea5-4295-a25f-547841d9fedb None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:57.310271 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ea3cb-8ea5-4295-a25f-547841d9fedb None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:57.310534 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ea3cb-8ea5-4295-a25f-547841d9fedb None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:57.311195 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ea3cb-8ea5-4295-a25f-547841d9fedb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:57.312549 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ea3cb-8ea5-4295-a25f-547841d9fedb None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:57.329073 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 474/918] 10.222.0.36 () {60 vars in 1141 bytes} [Fri Sep 2 08:56:57 2022] POST /identity/v3/services => generated 241 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:56:57.983412 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2c4c2-bae3-42ab-b415-5ea5186424dc None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:57.983938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2c4c2-bae3-42ab-b415-5ea5186424dc None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:57.983938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2c4c2-bae3-42ab-b415-5ea5186424dc None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:57.985548 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 445/919] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:57.990578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe886a-4749-4f20-af38-f0bb32a65db0 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:57.990984 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe886a-4749-4f20-af38-f0bb32a65db0 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:57.991148 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe886a-4749-4f20-af38-f0bb32a65db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:58.010105 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-5dbe886a-4749-4f20-af38-f0bb32a65db0 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:58.011527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5dbe886a-4749-4f20-af38-f0bb32a65db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:58.021595 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:58.088473 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 475/920] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:56:58.356614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2c4c2-bae3-42ab-b415-5ea5186424dc None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:58.384808 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:58.471853 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2c4c2-bae3-42ab-b415-5ea5186424dc None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:58.506070 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154ab8e-c705-4847-a291-0da6f31dfde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:58.506702 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154ab8e-c705-4847-a291-0da6f31dfde7 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:58.506980 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154ab8e-c705-4847-a291-0da6f31dfde7 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:58.507292 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154ab8e-c705-4847-a291-0da6f31dfde7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:58.507876 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154ab8e-c705-4847-a291-0da6f31dfde7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:58.509235 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154ab8e-c705-4847-a291-0da6f31dfde7 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:58.518097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-5154ab8e-c705-4847-a291-0da6f31dfde7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 08:56:58.518097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:58.518097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:56:58.518097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:56:58.518097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:56:58.518097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:56:58.518097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:56:58.518097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:58.518097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 08:56:58.518097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:58.518097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:56:58.518097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:58.518097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:58.518097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:58.518097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:58.518097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:58.518097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:58.519098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:58.519098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:58.519098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:58.519098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:58.519098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:58.519098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:58.519098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:56:58.519098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:56:58.519098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:56:58.519098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:56:58.519098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:58.519098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:58.519098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:58.519098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:58.519098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:58.519098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:58.519961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:58.519961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:58.519961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:58.519961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:58.519961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:58.519961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:58.519961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:58.519961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:58.519961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:58.519961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:58.519961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:56:58.519961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:58.519961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 08:56:58.519961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:58.519961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 446/921] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:56:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:58.525314 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe886a-4749-4f20-af38-f0bb32a65db0 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:58.628971 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe886a-4749-4f20-af38-f0bb32a65db0 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:58.645971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bf66e-6d0c-43b1-9c4e-553063160065 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:58.646249 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bf66e-6d0c-43b1-9c4e-553063160065 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:58.646510 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bf66e-6d0c-43b1-9c4e-553063160065 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:58.651745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 447/922] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:56:58.653160 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ecd94c-f7c4-4769-8a73-cd13cbb6a31d None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:58.653509 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ecd94c-f7c4-4769-8a73-cd13cbb6a31d None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:58.653873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ecd94c-f7c4-4769-8a73-cd13cbb6a31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:58.670925 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0678930-5cb2-4678-8c58-fb3d4a4cf80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:58.671625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0678930-5cb2-4678-8c58-fb3d4a4cf80b None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:58.671903 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0678930-5cb2-4678-8c58-fb3d4a4cf80b None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:58.672169 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0678930-5cb2-4678-8c58-fb3d4a4cf80b None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:58.672791 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0678930-5cb2-4678-8c58-fb3d4a4cf80b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:58.673087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0678930-5cb2-4678-8c58-fb3d4a4cf80b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:58.675021 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0678930-5cb2-4678-8c58-fb3d4a4cf80b None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:58.676211 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-36ecd94c-f7c4-4769-8a73-cd13cbb6a31d None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:58.677594 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-36ecd94c-f7c4-4769-8a73-cd13cbb6a31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:58.680726 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 476/923] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:56:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:58.687437 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0678930-5cb2-4678-8c58-fb3d4a4cf80b None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:58.693072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:58.768212 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 448/924] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 08:56:58.807799 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0678930-5cb2-4678-8c58-fb3d4a4cf80b None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:58.843120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28531c54-92d9-49ec-88b3-cb09ce0516fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:58.844058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28531c54-92d9-49ec-88b3-cb09ce0516fb None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:58.844584 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28531c54-92d9-49ec-88b3-cb09ce0516fb None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:58.845048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28531c54-92d9-49ec-88b3-cb09ce0516fb None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:58.846422 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28531c54-92d9-49ec-88b3-cb09ce0516fb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:58.846920 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28531c54-92d9-49ec-88b3-cb09ce0516fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:58.849239 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28531c54-92d9-49ec-88b3-cb09ce0516fb None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:58.855108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 477/925] 10.222.0.36 () {56 vars in 1114 bytes} [Fri Sep 2 08:56:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:56:58.864955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ecd94c-f7c4-4769-8a73-cd13cbb6a31d None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:58.891498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:58.969414 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ecd94c-f7c4-4769-8a73-cd13cbb6a31d None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:59.004816 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39d10a-0a0d-4001-8dc7-64e2a3c679f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:59.005544 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39d10a-0a0d-4001-8dc7-64e2a3c679f9 None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:59.005768 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39d10a-0a0d-4001-8dc7-64e2a3c679f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:59.005991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39d10a-0a0d-4001-8dc7-64e2a3c679f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:59.006641 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39d10a-0a0d-4001-8dc7-64e2a3c679f9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:59.008034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39d10a-0a0d-4001-8dc7-64e2a3c679f9 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:59.031176 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 449/926] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 08:56:58 2022] POST /identity/v3/endpoints => generated 346 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:56:59.047815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28531c54-92d9-49ec-88b3-cb09ce0516fb None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:59.133958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:59.174623 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28531c54-92d9-49ec-88b3-cb09ce0516fb None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:59.217104 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e074a7-0a26-4c7e-b961-56e874b9eedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:59.217794 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e074a7-0a26-4c7e-b961-56e874b9eedd None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:59.218080 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e074a7-0a26-4c7e-b961-56e874b9eedd None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:59.218351 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e074a7-0a26-4c7e-b961-56e874b9eedd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:59.230659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e074a7-0a26-4c7e-b961-56e874b9eedd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:56:59.235614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-50e074a7-0a26-4c7e-b961-56e874b9eedd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 08:56:59.235614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:59.235614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:59.235614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:59.235614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:59.235614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:59.235614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:59.235614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:59.235614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:59.235614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:59.235614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:59.235614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:59.235614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:56:59.235614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:56:59.235614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:56:59.235614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:56:59.236594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:56:59.236594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:56:59.236594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:56:59.236594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:56:59.236594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:59.236594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:59.236594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:59.236594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:59.236594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:59.236594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:59.236594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:59.236594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:59.236594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:59.236594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:59.236594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:59.236594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:59.237441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:59.237441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:59.237441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:59.237441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:59.237441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:56:59.237441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:56:59.237441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:56:59.237441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:56:59.237441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:56:59.237441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:56:59.237441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 08:56:59.237441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:59.237441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 478/927] 10.222.0.36 () {56 vars in 1120 bytes} [Fri Sep 2 08:56:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:59.242799 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39d10a-0a0d-4001-8dc7-64e2a3c679f9 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:59.252773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18d332-b351-490a-9619-4e85136758cc None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:59.253053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18d332-b351-490a-9619-4e85136758cc None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:59.253317 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18d332-b351-490a-9619-4e85136758cc None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:59.254768 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 479/928] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:56:59.259661 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d3156-b113-4079-a946-720179003e5e None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:59.259943 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d3156-b113-4079-a946-720179003e5e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:59.260217 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d3156-b113-4079-a946-720179003e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:59.278751 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-8c7d3156-b113-4079-a946-720179003e5e None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:59.279953 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8c7d3156-b113-4079-a946-720179003e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:59.324836 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:59.351620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39d10a-0a0d-4001-8dc7-64e2a3c679f9 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:59.363452 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 480/929] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 08:56:59.363452 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497057ca-d0d0-4782-815f-50b81f1a7fb9 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:59.363452 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497057ca-d0d0-4782-815f-50b81f1a7fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:59.363452 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497057ca-d0d0-4782-815f-50b81f1a7fb9 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:59.363994 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 481/930] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:56:59.372933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a46a5e-7663-441a-a385-8f990235d9d2 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:59.373309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a46a5e-7663-441a-a385-8f990235d9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:59.373832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a46a5e-7663-441a-a385-8f990235d9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:59.395651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca329f0-cca7-4bb0-a61a-d0015926748e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:59.396571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca329f0-cca7-4bb0-a61a-d0015926748e None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:59.396777 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-35a46a5e-7663-441a-a385-8f990235d9d2 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:56:59.397200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca329f0-cca7-4bb0-a61a-d0015926748e None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:59.397640 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca329f0-cca7-4bb0-a61a-d0015926748e None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:59.398016 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-35a46a5e-7663-441a-a385-8f990235d9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:56:59.398682 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca329f0-cca7-4bb0-a61a-d0015926748e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:59.399199 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca329f0-cca7-4bb0-a61a-d0015926748e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:59.400760 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca329f0-cca7-4bb0-a61a-d0015926748e None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:59.409933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 450/931] 10.222.0.36 () {56 vars in 1115 bytes} [Fri Sep 2 08:56:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:59.415271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca329f0-cca7-4bb0-a61a-d0015926748e None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:59.453441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:59.473332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 482/932] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:56:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:56:59.527405 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca329f0-cca7-4bb0-a61a-d0015926748e None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:59.561689 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371f8d7-84bb-46da-8fe4-aa97365f0ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:59.562621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371f8d7-84bb-46da-8fe4-aa97365f0ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:59.563095 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371f8d7-84bb-46da-8fe4-aa97365f0ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:59.563540 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7371f8d7-84bb-46da-8fe4-aa97365f0ee1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:59.575010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371f8d7-84bb-46da-8fe4-aa97365f0ee1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:59.576773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:59.577212 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7371f8d7-84bb-46da-8fe4-aa97365f0ee1 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:59.592642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-7371f8d7-84bb-46da-8fe4-aa97365f0ee1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:59.592642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:59.592642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:59.592642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:59.592642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:59.592642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:59.592642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:59.592642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:59.592642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:59.592642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:59.592642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:59.592642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:59.592642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:56:59.592642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:56:59.592642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:56:59.592642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:56:59.593582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:59.593582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:59.593582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:59.593582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:59.593582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:59.593582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:59.593582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:59.593582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:59.593582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:59.593582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:59.593582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:59.593582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:59.593582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:59.593582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:59.593582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:59.593582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:59.593582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:56:59.594482 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:56:59.594482 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:56:59.594482 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:59.596271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 451/933] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:56:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:59.604181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a46a5e-7663-441a-a385-8f990235d9d2 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:59.634092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371f8d7-84bb-46da-8fe4-aa97365f0ee1 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:59.705953 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a46a5e-7663-441a-a385-8f990235d9d2 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:59.738630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7371f8d7-84bb-46da-8fe4-aa97365f0ee1 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:59.741383 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f75f0-72a4-4894-9e0b-0ac6f2d61e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:59.742180 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f75f0-72a4-4894-9e0b-0ac6f2d61e35 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:59.742669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f75f0-72a4-4894-9e0b-0ac6f2d61e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:59.743178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f75f0-72a4-4894-9e0b-0ac6f2d61e35 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:59.744012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f75f0-72a4-4894-9e0b-0ac6f2d61e35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:56:59.744507 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f75f0-72a4-4894-9e0b-0ac6f2d61e35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:59.746074 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f75f0-72a4-4894-9e0b-0ac6f2d61e35 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:59.758991 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 483/934] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:56:59.764747 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f75f0-72a4-4894-9e0b-0ac6f2d61e35 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:59.775512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0050428-6963-4238-8f68-0a9b592a328d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:59.776143 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0050428-6963-4238-8f68-0a9b592a328d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:59.776418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0050428-6963-4238-8f68-0a9b592a328d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:59.776689 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0050428-6963-4238-8f68-0a9b592a328d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:59.777270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0050428-6963-4238-8f68-0a9b592a328d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:59.778614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0050428-6963-4238-8f68-0a9b592a328d None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:59.787589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-c0050428-6963-4238-8f68-0a9b592a328d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 08:56:59.787589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:59.787589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:56:59.787589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:56:59.787589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:56:59.787589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:56:59.787589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:56:59.787589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:59.787589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 08:56:59.787589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:59.787589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:56:59.787589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:59.787589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:59.787589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:59.787589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:59.787589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:59.787589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:59.788655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:59.788655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:59.788655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:59.788655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:59.788655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:59.788655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:59.788655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:56:59.788655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:56:59.788655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:56:59.788655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:56:59.788655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:59.788655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:59.788655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:59.788655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:59.788655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:59.788655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:59.789505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:59.789505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:59.789505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:59.789505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:59.789505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:59.789505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:59.789505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:59.789505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:59.789505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:59.789505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:59.789505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:56:59.789505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:59.789505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 08:56:59.789505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:59.793691 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 452/935] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:56:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:59.794503 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0050428-6963-4238-8f68-0a9b592a328d None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:59.821661 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:59.852665 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:56:59.880917 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f75f0-72a4-4894-9e0b-0ac6f2d61e35 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:59.903037 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0050428-6963-4238-8f68-0a9b592a328d None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:56:59.915973 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac23a7-2500-44da-8d32-b35eed74216c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:59.916597 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac23a7-2500-44da-8d32-b35eed74216c None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:59.916875 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac23a7-2500-44da-8d32-b35eed74216c None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:59.917423 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac23a7-2500-44da-8d32-b35eed74216c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:59.924710 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac23a7-2500-44da-8d32-b35eed74216c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:59.926142 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac23a7-2500-44da-8d32-b35eed74216c None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:59.937292 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33023f4-af3b-4b4e-9104-c30b09f81b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:56:59.937963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33023f4-af3b-4b4e-9104-c30b09f81b13 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:56:59.938147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-aeac23a7-2500-44da-8d32-b35eed74216c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 08:56:59.938147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:59.938147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:59.938147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:59.938147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:59.938147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:59.938147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:59.938147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:59.938147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:59.938147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:59.938147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:59.938147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:59.938147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:56:59.938147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:56:59.938147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:56:59.938147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:56:59.939095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:59.939095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:59.939095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:56:59.939095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:59.939095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:56:59.939095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:56:59.939095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:59.939095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:59.939095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:59.939095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:59.939095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:59.939095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:59.939095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:59.939095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:59.939095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:59.939095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:59.939095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:59.939095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:59.945379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:59.945379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:59.945379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:56:59.945379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:56:59.945379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:56:59.945379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:56:59.945379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:56:59.945379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:56:59.945379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 08:56:59.945379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:56:59.945914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33023f4-af3b-4b4e-9104-c30b09f81b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:56:59.945914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33023f4-af3b-4b4e-9104-c30b09f81b13 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:56:59.945914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33023f4-af3b-4b4e-9104-c30b09f81b13 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:56:59.946113 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 484/936] 10.222.0.36 () {56 vars in 1117 bytes} [Fri Sep 2 08:56:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:56:59.946113 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac23a7-2500-44da-8d32-b35eed74216c None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:56:59.946253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33023f4-af3b-4b4e-9104-c30b09f81b13 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:56:59.955532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-a33023f4-af3b-4b4e-9104-c30b09f81b13 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 08:56:59.955532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:59.955532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:56:59.955532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:56:59.955532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:56:59.955532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:56:59.955532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:56:59.955532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:59.955532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 08:56:59.955532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:59.955532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:56:59.955532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:59.955532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:56:59.955532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:56:59.955532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:56:59.955532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:56:59.955532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:56:59.956600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:56:59.956600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:56:59.956600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:56:59.956600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:56:59.956600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:56:59.956600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:56:59.956600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:56:59.956600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:56:59.956600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:56:59.956600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:56:59.956600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:56:59.956600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:56:59.956600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:56:59.956600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:56:59.956600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:56:59.956600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:56:59.962439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:56:59.962439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:56:59.962439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:56:59.962439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:56:59.962439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:56:59.962439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:56:59.962439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:56:59.962439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:56:59.962439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:56:59.962439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:56:59.962439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:56:59.962439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:56:59.962439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 08:56:59.962439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:56:59.962439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 453/937] 10.222.0.36 () {56 vars in 1126 bytes} [Fri Sep 2 08:56:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:56:59.964044 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33023f4-af3b-4b4e-9104-c30b09f81b13 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:00.053559 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac23a7-2500-44da-8d32-b35eed74216c None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:00.056060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:00.061323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33023f4-af3b-4b4e-9104-c30b09f81b13 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:00.077732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:00.092874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d8598-84c3-43a1-b687-54252136a340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:00.093481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d8598-84c3-43a1-b687-54252136a340 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:00.093760 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d8598-84c3-43a1-b687-54252136a340 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:00.094029 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d8598-84c3-43a1-b687-54252136a340 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:00.094654 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d8598-84c3-43a1-b687-54252136a340 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:00.094950 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d8598-84c3-43a1-b687-54252136a340 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:00.096391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d8598-84c3-43a1-b687-54252136a340 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:00.097443 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a2861-454a-47cc-b4b2-875e0bc7d53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:00.098049 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a2861-454a-47cc-b4b2-875e0bc7d53b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:00.098322 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a2861-454a-47cc-b4b2-875e0bc7d53b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:00.098590 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a2861-454a-47cc-b4b2-875e0bc7d53b None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:00.099336 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a2861-454a-47cc-b4b2-875e0bc7d53b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:00.099640 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a2861-454a-47cc-b4b2-875e0bc7d53b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:00.101032 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a2861-454a-47cc-b4b2-875e0bc7d53b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:00.101408 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 485/938] 10.222.0.36 () {56 vars in 1112 bytes} [Fri Sep 2 08:56:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:00.102184 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d8598-84c3-43a1-b687-54252136a340 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:00.123581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 454/939] 10.222.0.36 () {56 vars in 1149 bytes} [Fri Sep 2 08:56:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:00.123581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a2861-454a-47cc-b4b2-875e0bc7d53b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:00.221720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d8598-84c3-43a1-b687-54252136a340 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:00.227373 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a2861-454a-47cc-b4b2-875e0bc7d53b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:00.256300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5c998-f34b-421f-afc0-b144cd817993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:00.256903 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5c998-f34b-421f-afc0-b144cd817993 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:00.257178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5c998-f34b-421f-afc0-b144cd817993 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:00.257441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e5c998-f34b-421f-afc0-b144cd817993 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:00.258167 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5c998-f34b-421f-afc0-b144cd817993 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:00.258576 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5c998-f34b-421f-afc0-b144cd817993 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:00.260039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e5c998-f34b-421f-afc0-b144cd817993 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:00.263193 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962fae6-1b16-4218-9345-b2d0fd1bbb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:00.263793 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962fae6-1b16-4218-9345-b2d0fd1bbb83 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:00.264071 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962fae6-1b16-4218-9345-b2d0fd1bbb83 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:00.264340 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962fae6-1b16-4218-9345-b2d0fd1bbb83 None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:00.264959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962fae6-1b16-4218-9345-b2d0fd1bbb83 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:00.265308 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 486/940] 10.222.0.36 () {56 vars in 1122 bytes} [Fri Sep 2 08:57:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:00.266165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5c998-f34b-421f-afc0-b144cd817993 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:00.267993 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962fae6-1b16-4218-9345-b2d0fd1bbb83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:00.270752 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962fae6-1b16-4218-9345-b2d0fd1bbb83 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:00.275978 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 455/941] 10.222.0.36 () {56 vars in 1112 bytes} [Fri Sep 2 08:57:00 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:00.279823 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962fae6-1b16-4218-9345-b2d0fd1bbb83 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:00.300168 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:00.342067 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:00.397199 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e5c998-f34b-421f-afc0-b144cd817993 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:00.424320 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962fae6-1b16-4218-9345-b2d0fd1bbb83 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:00.448483 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0e08d-29d4-4082-a4b0-06e29e170f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:00.449263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0e08d-29d4-4082-a4b0-06e29e170f0e None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:00.449542 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0e08d-29d4-4082-a4b0-06e29e170f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:00.449816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0e08d-29d4-4082-a4b0-06e29e170f0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:00.466822 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c6165-ed8b-404a-bbd6-e25b9f11282a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:00.467685 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c6165-ed8b-404a-bbd6-e25b9f11282a None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:00.468463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c6165-ed8b-404a-bbd6-e25b9f11282a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:00.468932 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c6165-ed8b-404a-bbd6-e25b9f11282a None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:00.470017 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c6165-ed8b-404a-bbd6-e25b9f11282a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:00.470132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a0e08d-29d4-4082-a4b0-06e29e170f0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:00.470593 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c6165-ed8b-404a-bbd6-e25b9f11282a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:00.471899 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a0e08d-29d4-4082-a4b0-06e29e170f0e None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:00.472314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c6165-ed8b-404a-bbd6-e25b9f11282a None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:00.478471 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 456/942] 10.222.0.36 () {56 vars in 1122 bytes} [Fri Sep 2 08:57:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:00.487090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c6165-ed8b-404a-bbd6-e25b9f11282a None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:00.494091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-07a0e08d-29d4-4082-a4b0-06e29e170f0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:57:00.494091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:00.494091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:00.494091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:00.494091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:00.494091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:00.494091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:00.494091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:00.494091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:00.494091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:00.494091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:00.494091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:00.494091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:57:00.494091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:57:00.494091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:57:00.494091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:57:00.495062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:00.495062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:00.495062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:00.495062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:00.495062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:00.495062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:00.495062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:00.495062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:00.495062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:00.495062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:00.495062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:00.495062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:00.495062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:00.495062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:00.495062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:00.495062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:00.495062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:57:00.496327 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:57:00.496327 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:57:00.496327 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:00.496327 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 487/943] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:57:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:00.502645 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0e08d-29d4-4082-a4b0-06e29e170f0e None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:00.606086 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c6165-ed8b-404a-bbd6-e25b9f11282a None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:00.608149 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:00.608894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:00.628388 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0e08d-29d4-4082-a4b0-06e29e170f0e None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:00.655446 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04721e-6086-4eac-8477-b454ee350b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:00.656706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04721e-6086-4eac-8477-b454ee350b23 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:00.657392 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04721e-6086-4eac-8477-b454ee350b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:00.657840 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04721e-6086-4eac-8477-b454ee350b23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:00.658942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c04721e-6086-4eac-8477-b454ee350b23 None admin] RBAC: Adding query filter params (interface=public, service_id=ca24d292b70d40ababa408b83d3e59f9, region_id=RegionOne) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:00.659834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c04721e-6086-4eac-8477-b454ee350b23 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:00.662012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c04721e-6086-4eac-8477-b454ee350b23 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:00.667953 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124a46b-f1f8-4ac9-ac42-365163bf7e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:00.668633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 457/944] 10.222.0.36 () {56 vars in 1251 bytes} [Fri Sep 2 08:57:00 2022] GET /identity/v3/endpoints?service_id=ca24d292b70d40ababa408b83d3e59f9&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:00.668753 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124a46b-f1f8-4ac9-ac42-365163bf7e9e None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:00.673430 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124a46b-f1f8-4ac9-ac42-365163bf7e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:00.674307 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124a46b-f1f8-4ac9-ac42-365163bf7e9e None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:00.674927 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04721e-6086-4eac-8477-b454ee350b23 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:00.675033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124a46b-f1f8-4ac9-ac42-365163bf7e9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:00.675466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124a46b-f1f8-4ac9-ac42-365163bf7e9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:00.677145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124a46b-f1f8-4ac9-ac42-365163bf7e9e None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:00.696121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 488/945] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:00.697414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124a46b-f1f8-4ac9-ac42-365163bf7e9e None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:00.805028 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124a46b-f1f8-4ac9-ac42-365163bf7e9e None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:00.816525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04721e-6086-4eac-8477-b454ee350b23 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:00.841051 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03ed4f-1d0b-4587-8a7b-1f9d74e35ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:00.841683 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03ed4f-1d0b-4587-8a7b-1f9d74e35ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:00.842003 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03ed4f-1d0b-4587-8a7b-1f9d74e35ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:00.842284 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03ed4f-1d0b-4587-8a7b-1f9d74e35ba4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:00.849964 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f03ed4f-1d0b-4587-8a7b-1f9d74e35ba4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:00.851422 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f03ed4f-1d0b-4587-8a7b-1f9d74e35ba4 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:00.859978 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e190b09-a77a-4ee8-b30c-741728fb5443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:00.860840 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e190b09-a77a-4ee8-b30c-741728fb5443 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:00.861928 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e190b09-a77a-4ee8-b30c-741728fb5443 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:00.862558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e190b09-a77a-4ee8-b30c-741728fb5443 None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:00.863393 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e190b09-a77a-4ee8-b30c-741728fb5443 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:00.864039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-7f03ed4f-1d0b-4587-8a7b-1f9d74e35ba4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:57:00.864039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:00.864039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:00.864039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:00.864039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:00.864039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:00.864039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:00.864039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:00.864039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:00.864039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:00.864039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:00.864039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:00.864039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:57:00.864039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:57:00.864039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:57:00.864906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:57:00.864906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:00.864906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:00.864906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:00.864906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:00.864906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:00.864906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:00.864906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:00.864906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:00.864906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:00.864906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:00.864906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:00.864906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:00.864906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:00.864906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:00.864906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:00.864906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:00.865812 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:57:00.865812 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:57:00.865812 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:57:00.865812 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:57:00.865812 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:57:00.865812 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:57:00.865812 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:57:00.865812 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:00.867738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e190b09-a77a-4ee8-b30c-741728fb5443 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:00.870066 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 489/946] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:57:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:00.870914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03ed4f-1d0b-4587-8a7b-1f9d74e35ba4 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:00.875637 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:00.879836 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 458/947] 10.222.0.36 () {56 vars in 1087 bytes} [Fri Sep 2 08:57:00 2022] GET /identity/v3/services => generated 1497 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 08:57:00.994340 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03ed4f-1d0b-4587-8a7b-1f9d74e35ba4 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:01.031709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758d2ec-eb35-45d5-96dc-24f6c775235d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:01.032368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758d2ec-eb35-45d5-96dc-24f6c775235d None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:01.032646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758d2ec-eb35-45d5-96dc-24f6c775235d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:01.033540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758d2ec-eb35-45d5-96dc-24f6c775235d None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:01.034309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758d2ec-eb35-45d5-96dc-24f6c775235d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:01.034616 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758d2ec-eb35-45d5-96dc-24f6c775235d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:01.035562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986e068-c238-4d8e-945f-95d7dbfe8344 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:01.035856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986e068-c238-4d8e-945f-95d7dbfe8344 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:01.036071 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758d2ec-eb35-45d5-96dc-24f6c775235d None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:01.036172 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3986e068-c238-4d8e-945f-95d7dbfe8344 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:01.037583 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 459/948] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:01.042788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f891e5-7927-48f7-b8b0-c9cdc3d1b579 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:01.043085 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f891e5-7927-48f7-b8b0-c9cdc3d1b579 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:01.043442 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f891e5-7927-48f7-b8b0-c9cdc3d1b579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:01.049567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 490/949] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:57:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:01.055433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758d2ec-eb35-45d5-96dc-24f6c775235d None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:01.068564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-05f891e5-7927-48f7-b8b0-c9cdc3d1b579 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:01.070860 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-05f891e5-7927-48f7-b8b0-c9cdc3d1b579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:01.105802 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:01.131239 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:01.163852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 460/950] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:01.186825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758d2ec-eb35-45d5-96dc-24f6c775235d None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:01.222716 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9c76e-e53c-4fb7-a726-2d2622580c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:01.223406 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9c76e-e53c-4fb7-a726-2d2622580c68 None admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:01.223685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9c76e-e53c-4fb7-a726-2d2622580c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:01.223959 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9c76e-e53c-4fb7-a726-2d2622580c68 None admin] PATH_INFO: `/v3/projects/eef80493db09437b88f2b147a59928c1/users/76ebecce283f491e82a961d7f739878f/roles/a604599bb8af4abfaada3a92ac76b1bd` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:01.226311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa9c76e-e53c-4fb7-a726-2d2622580c68 None admin] RBAC: Authorizing `identity:create_grant(project_id=eef80493db09437b88f2b147a59928c1, user_id=76ebecce283f491e82a961d7f739878f, role_id=a604599bb8af4abfaada3a92ac76b1bd)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:01.227845 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa9c76e-e53c-4fb7-a726-2d2622580c68 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:01.242401 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 491/951] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:57:01 2022] PUT /identity/v3/projects/eef80493db09437b88f2b147a59928c1/users/76ebecce283f491e82a961d7f739878f/roles/a604599bb8af4abfaada3a92ac76b1bd => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 08:57:01.469101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f891e5-7927-48f7-b8b0-c9cdc3d1b579 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:01.571799 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f891e5-7927-48f7-b8b0-c9cdc3d1b579 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:01.611251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa7545-fd6f-4788-9556-a83ef8c699fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:01.612196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa7545-fd6f-4788-9556-a83ef8c699fb None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:01.612688 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa7545-fd6f-4788-9556-a83ef8c699fb None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:01.613162 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa7545-fd6f-4788-9556-a83ef8c699fb None admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:01.614004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa7545-fd6f-4788-9556-a83ef8c699fb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:01.615647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa7545-fd6f-4788-9556-a83ef8c699fb None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:01.618503 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: WARNING keystone.server.flask.common [None req-a7fa7545-fd6f-4788-9556-a83ef8c699fb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 02 08:57:01.643287 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.sql.core [None req-a7fa7545-fd6f-4788-9556-a83ef8c699fb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 02 08:57:01.643287 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 02 08:57:01.643287 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [parameters: {'id': 'f2b9ce4a262a415facd9448ea223354d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 02 08:57:01.643287 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81390) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 02 08:57:01.660438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-a7fa7545-fd6f-4788-9556-a83ef8c699fb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f2b9ce4a262a415facd9448ea223354d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f2b9ce4a262a415facd9448ea223354d. Sep 02 08:57:01.660438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:01.660438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 02 08:57:01.660438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 02 08:57:01.660438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 02 08:57:01.660438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 02 08:57:01.660438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 02 08:57:01.660438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application result = self._query(query) Sep 02 08:57:01.660438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 02 08:57:01.660438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application conn.query(q) Sep 02 08:57:01.660438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 02 08:57:01.660438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 02 08:57:01.660438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 02 08:57:01.661289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application result.read() Sep 02 08:57:01.661289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 02 08:57:01.661289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 02 08:57:01.661289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 02 08:57:01.661289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 02 08:57:01.661289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 02 08:57:01.661289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 02 08:57:01.661289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 02 08:57:01.661289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 02 08:57:01.661289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 02 08:57:01.661289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:01.661289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 02 08:57:01.661289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:01.661289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:01.661289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 02 08:57:01.661289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 02 08:57:01.661289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 02 08:57:01.661289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 02 08:57:01.662281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 02 08:57:01.662281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application next(self.gen) Sep 02 08:57:01.662281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 02 08:57:01.662281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application yield resource Sep 02 08:57:01.662281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 02 08:57:01.662281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application next(self.gen) Sep 02 08:57:01.662281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 02 08:57:01.662281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self.session.rollback() Sep 02 08:57:01.662281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 02 08:57:01.662281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self.force_reraise() Sep 02 08:57:01.662281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 02 08:57:01.662281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise self.value Sep 02 08:57:01.662281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 02 08:57:01.662281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 02 08:57:01.662281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 02 08:57:01.662281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application session.commit() Sep 02 08:57:01.662281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 02 08:57:01.663745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 02 08:57:01.663745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 02 08:57:01.663745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._prepare_impl() Sep 02 08:57:01.663745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 02 08:57:01.663745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self.session.flush() Sep 02 08:57:01.663745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 02 08:57:01.663745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._flush(objects) Sep 02 08:57:01.663745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 02 08:57:01.663745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 02 08:57:01.663745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 02 08:57:01.663745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application compat.raise_( Sep 02 08:57:01.663745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 02 08:57:01.663745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception Sep 02 08:57:01.663745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 02 08:57:01.663745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application flush_context.execute() Sep 02 08:57:01.663745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 02 08:57:01.663745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rec.execute(self) Sep 02 08:57:01.664661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 02 08:57:01.664661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 02 08:57:01.664661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 02 08:57:01.664661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 02 08:57:01.664661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 02 08:57:01.664661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 02 08:57:01.664661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 02 08:57:01.664661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 02 08:57:01.664661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 02 08:57:01.664661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 02 08:57:01.664661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 02 08:57:01.664661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 02 08:57:01.664661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 02 08:57:01.664661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 02 08:57:01.664661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 02 08:57:01.665452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 02 08:57:01.665452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 02 08:57:01.665452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception Sep 02 08:57:01.665452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 02 08:57:01.665452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 02 08:57:01.665452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 02 08:57:01.665452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 02 08:57:01.665452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 02 08:57:01.665452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application result = self._query(query) Sep 02 08:57:01.665452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 02 08:57:01.665452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application conn.query(q) Sep 02 08:57:01.665452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 02 08:57:01.665452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 02 08:57:01.665452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 02 08:57:01.665452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application result.read() Sep 02 08:57:01.665452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 02 08:57:01.665452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 02 08:57:01.666344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 02 08:57:01.666344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 02 08:57:01.666344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 02 08:57:01.666344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 02 08:57:01.666344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 02 08:57:01.666344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 02 08:57:01.666344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 02 08:57:01.666344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 02 08:57:01.666344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [parameters: {'id': 'f2b9ce4a262a415facd9448ea223354d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 02 08:57:01.666344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 02 08:57:01.666344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:01.666344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:57:01.666344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:01.666344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:01.666344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 02 08:57:01.667188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 02 08:57:01.667188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 02 08:57:01.667188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 02 08:57:01.667188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 02 08:57:01.667188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:01.667188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:57:01.667188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:01.667188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:01.667188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:01.667188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:01.667188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:01.667188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:01.667188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:01.667188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:01.667188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:01.667188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:01.667188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:01.669558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:01.669558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 02 08:57:01.669558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 02 08:57:01.669558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:01.669558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:01.669558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 02 08:57:01.669558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 02 08:57:01.669558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f2b9ce4a262a415facd9448ea223354d. Sep 02 08:57:01.669558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:01.672799 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 461/952] 10.222.0.36 () {60 vars in 1140 bytes} [Fri Sep 2 08:57:01 2022] POST /identity/v3/projects => generated 262 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:01.678799 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9c76e-e53c-4fb7-a726-2d2622580c68 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:01.787542 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9c76e-e53c-4fb7-a726-2d2622580c68 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:01.813002 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:01.826407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d9923-8ab1-4c7e-a38e-42a9fbecdfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:01.827105 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d9923-8ab1-4c7e-a38e-42a9fbecdfb8 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:01.827425 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d9923-8ab1-4c7e-a38e-42a9fbecdfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:01.827698 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d9923-8ab1-4c7e-a38e-42a9fbecdfb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:01.835473 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d9923-8ab1-4c7e-a38e-42a9fbecdfb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:01.836927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93d9923-8ab1-4c7e-a38e-42a9fbecdfb8 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:01.848294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-c93d9923-8ab1-4c7e-a38e-42a9fbecdfb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:57:01.848294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:01.848294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:01.848294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:01.848294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:01.848294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:01.848294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:01.848294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:01.848294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:01.848294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:01.848294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:01.848294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:01.848294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:57:01.848294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:57:01.848294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:57:01.850403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:57:01.850403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:01.850403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:01.850403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:01.850403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:01.850403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:01.850403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:01.850403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:01.850403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:01.850403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:01.850403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:01.850403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:01.850403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:01.850403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:01.850403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:01.850403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:01.850403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:01.852644 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:57:01.852644 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:57:01.852644 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:57:01.852644 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:57:01.852644 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:57:01.852644 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:57:01.852644 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:57:01.852644 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:01.852644 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 492/953] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:57:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:01.859942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa7545-fd6f-4788-9556-a83ef8c699fb None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:01.880122 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:01.965987 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa7545-fd6f-4788-9556-a83ef8c699fb None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:02.004437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d80ca1-de43-4d58-b80e-80f26f8e6f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:02.005416 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d80ca1-de43-4d58-b80e-80f26f8e6f56 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:02.005911 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d80ca1-de43-4d58-b80e-80f26f8e6f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:02.006385 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d80ca1-de43-4d58-b80e-80f26f8e6f56 None admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:02.007273 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d80ca1-de43-4d58-b80e-80f26f8e6f56 None admin] RBAC: Adding query filter params (name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:02.007787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d80ca1-de43-4d58-b80e-80f26f8e6f56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:02.009600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d80ca1-de43-4d58-b80e-80f26f8e6f56 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:02.031624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 462/954] 10.222.0.36 () {56 vars in 1112 bytes} [Fri Sep 2 08:57:01 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:02.242785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c690e84-3b72-4eb7-8376-0d8792a9a19b None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:02.243312 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c690e84-3b72-4eb7-8376-0d8792a9a19b None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:02.243312 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c690e84-3b72-4eb7-8376-0d8792a9a19b None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:02.244858 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 493/955] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:02.250365 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf92f0-7afd-4ea4-bc7e-9205ed841d3d None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:02.250786 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf92f0-7afd-4ea4-bc7e-9205ed841d3d None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:02.250959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cf92f0-7afd-4ea4-bc7e-9205ed841d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:02.269498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-92cf92f0-7afd-4ea4-bc7e-9205ed841d3d None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:02.270998 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-92cf92f0-7afd-4ea4-bc7e-9205ed841d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:02.347098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 463/956] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:02.613925 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c690e84-3b72-4eb7-8376-0d8792a9a19b None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:02.715495 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c690e84-3b72-4eb7-8376-0d8792a9a19b None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:02.751426 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b059b-f2d7-4176-abb0-bfb1186dc11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:02.752099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b059b-f2d7-4176-abb0-bfb1186dc11a None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:02.752372 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b059b-f2d7-4176-abb0-bfb1186dc11a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:02.752640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b059b-f2d7-4176-abb0-bfb1186dc11a None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:02.753291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b059b-f2d7-4176-abb0-bfb1186dc11a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:02.754656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b059b-f2d7-4176-abb0-bfb1186dc11a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:02.772104 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 494/957] 10.222.0.36 () {60 vars in 1141 bytes} [Fri Sep 2 08:57:02 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:57:02.879999 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509133c-437f-4719-8665-b552ac6c6998 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:02.880308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509133c-437f-4719-8665-b552ac6c6998 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:02.880501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509133c-437f-4719-8665-b552ac6c6998 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:02.882177 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 464/958] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:02.887805 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb6f20-a543-4200-bd1c-343dcc742a44 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:02.888109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb6f20-a543-4200-bd1c-343dcc742a44 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:02.888354 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb6f20-a543-4200-bd1c-343dcc742a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:02.909053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-c7bb6f20-a543-4200-bd1c-343dcc742a44 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:02.910242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c7bb6f20-a543-4200-bd1c-343dcc742a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:02.952123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:02.990111 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 495/959] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:03.261527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509133c-437f-4719-8665-b552ac6c6998 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:03.270191 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b447a8-96d1-48ef-8d20-85cded78e687 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:03.271051 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b447a8-96d1-48ef-8d20-85cded78e687 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:03.271294 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:03.271779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b447a8-96d1-48ef-8d20-85cded78e687 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:03.273326 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 496/960] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:57:03.278645 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f307850-bc02-4c42-810d-2b310d52fbb0 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:03.278921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f307850-bc02-4c42-810d-2b310d52fbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:03.279216 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f307850-bc02-4c42-810d-2b310d52fbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:03.304528 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-8f307850-bc02-4c42-810d-2b310d52fbb0 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:03.306816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8f307850-bc02-4c42-810d-2b310d52fbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:03.386498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509133c-437f-4719-8665-b552ac6c6998 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:03.399447 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 497/961] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:03 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:03.420379 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae679654-d5f0-472f-b0d1-1ed7b887c26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:03.421236 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae679654-d5f0-472f-b0d1-1ed7b887c26a None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:03.421679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae679654-d5f0-472f-b0d1-1ed7b887c26a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:03.422107 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae679654-d5f0-472f-b0d1-1ed7b887c26a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:03.422836 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae679654-d5f0-472f-b0d1-1ed7b887c26a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:03.424914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae679654-d5f0-472f-b0d1-1ed7b887c26a None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:03.434139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-ae679654-d5f0-472f-b0d1-1ed7b887c26a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 08:57:03.434139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:03.434139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:57:03.434139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:57:03.434139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:57:03.434139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:57:03.434139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:57:03.434139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:57:03.434139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 08:57:03.434139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:03.434139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:57:03.434139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:03.434139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:03.434139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:03.434139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:03.434139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:03.434139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:03.435538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:03.435538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:03.435538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:03.435538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:03.435538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:03.435538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:03.435538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:57:03.435538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:57:03.435538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:57:03.435538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:57:03.435538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:03.435538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:03.435538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:03.435538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:03.435538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:03.435538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:03.436620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:03.436620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:03.436620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:03.436620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:03.436620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:03.436620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:03.436620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:03.436620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:03.436620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:03.436620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:03.436620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:57:03.436620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:57:03.436620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 08:57:03.436620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:03.438945 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 465/962] 10.222.0.36 () {56 vars in 1126 bytes} [Fri Sep 2 08:57:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:03.445164 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f307850-bc02-4c42-810d-2b310d52fbb0 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:03.547105 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:03.551618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f307850-bc02-4c42-810d-2b310d52fbb0 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:03.588470 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c652cf-0db0-493c-8924-17f7939a5871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:03.589408 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c652cf-0db0-493c-8924-17f7939a5871 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:03.589922 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c652cf-0db0-493c-8924-17f7939a5871 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:03.590398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c652cf-0db0-493c-8924-17f7939a5871 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:03.591309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c652cf-0db0-493c-8924-17f7939a5871 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:03.591810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c652cf-0db0-493c-8924-17f7939a5871 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:03.593385 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c652cf-0db0-493c-8924-17f7939a5871 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:03.598810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 498/963] 10.222.0.36 () {56 vars in 1122 bytes} [Fri Sep 2 08:57:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:03.604702 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae679654-d5f0-472f-b0d1-1ed7b887c26a None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:03.680105 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:03.681294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c652cf-0db0-493c-8924-17f7939a5871 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:03.726150 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae679654-d5f0-472f-b0d1-1ed7b887c26a None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:03.767914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cba30-91d3-4ebe-8e05-66064947d70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:03.768524 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cba30-91d3-4ebe-8e05-66064947d70d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:03.768793 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cba30-91d3-4ebe-8e05-66064947d70d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:03.769241 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97cba30-91d3-4ebe-8e05-66064947d70d None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:03.770031 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cba30-91d3-4ebe-8e05-66064947d70d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:03.770485 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cba30-91d3-4ebe-8e05-66064947d70d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:03.772038 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97cba30-91d3-4ebe-8e05-66064947d70d None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:03.777888 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 466/964] 10.222.0.36 () {56 vars in 1122 bytes} [Fri Sep 2 08:57:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:03.783918 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cba30-91d3-4ebe-8e05-66064947d70d None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:03.800669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c652cf-0db0-493c-8924-17f7939a5871 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:03.836826 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4cfe8-c2d1-40f7-88f4-c0d1a1e51c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:03.837842 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4cfe8-c2d1-40f7-88f4-c0d1a1e51c27 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:03.838400 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4cfe8-c2d1-40f7-88f4-c0d1a1e51c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:03.838883 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4cfe8-c2d1-40f7-88f4-c0d1a1e51c27 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:03.850300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee4cfe8-c2d1-40f7-88f4-c0d1a1e51c27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:57:03.855496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-fee4cfe8-c2d1-40f7-88f4-c0d1a1e51c27 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 08:57:03.855496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:03.855496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:03.855496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:03.855496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:03.855496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:03.855496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:03.855496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:03.855496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:03.855496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:03.855496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:03.855496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:03.855496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:57:03.855496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:57:03.855496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:57:03.855496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:57:03.856466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:57:03.856466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:57:03.856466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:57:03.856466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:57:03.856466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:03.856466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:03.856466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:03.856466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:03.856466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:03.856466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:03.856466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:03.856466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:03.856466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:03.856466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:03.856466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:03.856466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:03.857320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:03.857320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:03.857320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:03.857320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:03.857320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:57:03.857320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:57:03.857320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:57:03.857320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:57:03.857320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:57:03.857320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:57:03.857320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 08:57:03.857320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:03.859988 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 499/965] 10.222.0.36 () {56 vars in 1120 bytes} [Fri Sep 2 08:57:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:03.866525 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4cfe8-c2d1-40f7-88f4-c0d1a1e51c27 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:03.890579 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97cba30-91d3-4ebe-8e05-66064947d70d None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:03.895205 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:03.928774 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c569f-9d42-4165-a561-71ccc2cc7dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:03.929689 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c569f-9d42-4165-a561-71ccc2cc7dc5 None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:03.930120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c569f-9d42-4165-a561-71ccc2cc7dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:03.930534 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408c569f-9d42-4165-a561-71ccc2cc7dc5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:03.931396 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c569f-9d42-4165-a561-71ccc2cc7dc5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:03.932900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408c569f-9d42-4165-a561-71ccc2cc7dc5 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:03.938323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:03.961646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 467/966] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 08:57:03 2022] POST /identity/v3/endpoints => generated 349 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 08:57:03.999084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4cfe8-c2d1-40f7-88f4-c0d1a1e51c27 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:04.047291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce438910-64b3-4e41-8bcf-d7e1a0b1e9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:04.048903 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce438910-64b3-4e41-8bcf-d7e1a0b1e9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:04.049402 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce438910-64b3-4e41-8bcf-d7e1a0b1e9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:04.049889 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce438910-64b3-4e41-8bcf-d7e1a0b1e9d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:04.050816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce438910-64b3-4e41-8bcf-d7e1a0b1e9d5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:04.051387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce438910-64b3-4e41-8bcf-d7e1a0b1e9d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:04.052989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce438910-64b3-4e41-8bcf-d7e1a0b1e9d5 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:04.063535 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 500/967] 10.222.0.36 () {56 vars in 1115 bytes} [Fri Sep 2 08:57:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:04.072558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c569f-9d42-4165-a561-71ccc2cc7dc5 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:04.093495 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab15a9-552d-478a-b93a-e8ac8acabc5a None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:04.093787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab15a9-552d-478a-b93a-e8ac8acabc5a None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:04.094061 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab15a9-552d-478a-b93a-e8ac8acabc5a None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:04.096141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 501/968] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:04.102699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe08e4-80ef-4023-ac72-9ae0591c40b6 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:04.102984 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe08e4-80ef-4023-ac72-9ae0591c40b6 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:04.103309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe08e4-80ef-4023-ac72-9ae0591c40b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:04.124004 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-6cfe08e4-80ef-4023-ac72-9ae0591c40b6 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:04.125192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6cfe08e4-80ef-4023-ac72-9ae0591c40b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:04.193754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408c569f-9d42-4165-a561-71ccc2cc7dc5 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:04.202837 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 502/969] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:04 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 08:57:04.233192 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a09c4e-1fb6-4b4a-a7c0-bd63e7ffa19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:04.233835 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a09c4e-1fb6-4b4a-a7c0-bd63e7ffa19c None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:04.234116 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a09c4e-1fb6-4b4a-a7c0-bd63e7ffa19c None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:04.234390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a09c4e-1fb6-4b4a-a7c0-bd63e7ffa19c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:04.246344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a09c4e-1fb6-4b4a-a7c0-bd63e7ffa19c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:04.247839 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a09c4e-1fb6-4b4a-a7c0-bd63e7ffa19c None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:04.264289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-38a09c4e-1fb6-4b4a-a7c0-bd63e7ffa19c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:57:04.264289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:04.264289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:04.264289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:04.264289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:04.264289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:04.264289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:04.264289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:04.264289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:04.264289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:04.264289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:04.264289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:04.264289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:57:04.264289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:57:04.264289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:57:04.264289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:57:04.265767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:04.265767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:04.265767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:04.265767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:04.265767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:04.265767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:04.265767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:04.265767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:04.265767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:04.265767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:04.265767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:04.265767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:04.265767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:04.265767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:04.265767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:04.265767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:04.265767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:57:04.266681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:57:04.266681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:57:04.266681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:04.266681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 468/970] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:57:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:04.276403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe08e4-80ef-4023-ac72-9ae0591c40b6 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:04.292705 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:04.390984 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe08e4-80ef-4023-ac72-9ae0591c40b6 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:04.426965 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db7aba-a72a-4102-9fa9-a201d40a5561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:04.427599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db7aba-a72a-4102-9fa9-a201d40a5561 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:04.427878 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db7aba-a72a-4102-9fa9-a201d40a5561 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:04.428147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5db7aba-a72a-4102-9fa9-a201d40a5561 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:04.428785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db7aba-a72a-4102-9fa9-a201d40a5561 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:04.429080 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db7aba-a72a-4102-9fa9-a201d40a5561 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:04.430445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5db7aba-a72a-4102-9fa9-a201d40a5561 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:04.443523 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 503/971] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:04.449054 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a09c4e-1fb6-4b4a-a7c0-bd63e7ffa19c None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:04.484389 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db7aba-a72a-4102-9fa9-a201d40a5561 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:04.498697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:04.563105 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a09c4e-1fb6-4b4a-a7c0-bd63e7ffa19c None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:04.573186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:04.603131 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3c5c6-418f-432b-b421-7eface741a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:04.603779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3c5c6-418f-432b-b421-7eface741a26 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:04.604059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3c5c6-418f-432b-b421-7eface741a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:04.604332 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3c5c6-418f-432b-b421-7eface741a26 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:04.607148 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5db7aba-a72a-4102-9fa9-a201d40a5561 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:04.612665 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a3c5c6-418f-432b-b421-7eface741a26 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:04.614122 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a3c5c6-418f-432b-b421-7eface741a26 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:04.626931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-29a3c5c6-418f-432b-b421-7eface741a26 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 08:57:04.626931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:04.626931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:04.626931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:04.626931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:04.626931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:04.626931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:04.626931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:04.626931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:04.626931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:04.626931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:04.626931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:04.626931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:57:04.626931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:57:04.626931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:57:04.626931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:57:04.634565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:04.634565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:04.634565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:57:04.634565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:57:04.634565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:57:04.634565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:57:04.634565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:04.634565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:04.634565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:04.634565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:04.634565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:04.634565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:04.634565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:04.634565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:04.634565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:04.634565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:04.634565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:04.634565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:04.635644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:04.635644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:04.635644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:57:04.635644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:57:04.635644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:57:04.635644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:57:04.635644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:57:04.635644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:57:04.635644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 08:57:04.635644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:04.635644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 469/972] 10.222.0.36 () {56 vars in 1117 bytes} [Fri Sep 2 08:57:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:04.637495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3c5c6-418f-432b-b421-7eface741a26 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:04.651724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e61eb2-6044-4df4-b435-92fb8f20c282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:04.652669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e61eb2-6044-4df4-b435-92fb8f20c282 None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:04.653139 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e61eb2-6044-4df4-b435-92fb8f20c282 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:04.653606 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e61eb2-6044-4df4-b435-92fb8f20c282 None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:04.655108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e61eb2-6044-4df4-b435-92fb8f20c282 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:04.657889 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e61eb2-6044-4df4-b435-92fb8f20c282 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:04.662427 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: WARNING keystone.server.flask.common [None req-90e61eb2-6044-4df4-b435-92fb8f20c282 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 02 08:57:04.677084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.sql.core [None req-90e61eb2-6044-4df4-b435-92fb8f20c282 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 02 08:57:04.677084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 02 08:57:04.677084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [parameters: {'user_id': '7b11b125065642fdab68370f3e0ec9c4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 02 08:57:04.677084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81389) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 02 08:57:04.685238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-90e61eb2-6044-4df4-b435-92fb8f20c282 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 02 08:57:04.685238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:04.685238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 02 08:57:04.685238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 02 08:57:04.685238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 02 08:57:04.685238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 02 08:57:04.685238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 02 08:57:04.685238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application result = self._query(query) Sep 02 08:57:04.685238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 02 08:57:04.685238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application conn.query(q) Sep 02 08:57:04.685238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 02 08:57:04.685238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 02 08:57:04.685238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 02 08:57:04.685238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application result.read() Sep 02 08:57:04.685238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 02 08:57:04.687092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 02 08:57:04.687092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 02 08:57:04.687092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 02 08:57:04.687092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 02 08:57:04.687092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 02 08:57:04.687092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 02 08:57:04.687092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 02 08:57:04.687092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 02 08:57:04.687092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:04.687092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 02 08:57:04.687092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:04.687092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:04.687092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 02 08:57:04.687092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 02 08:57:04.687092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 02 08:57:04.687092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 02 08:57:04.687092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 02 08:57:04.687092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application next(self.gen) Sep 02 08:57:04.688296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 02 08:57:04.688296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application yield resource Sep 02 08:57:04.688296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 02 08:57:04.688296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application next(self.gen) Sep 02 08:57:04.688296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 02 08:57:04.688296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self.session.rollback() Sep 02 08:57:04.688296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 02 08:57:04.688296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self.force_reraise() Sep 02 08:57:04.688296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 02 08:57:04.688296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise self.value Sep 02 08:57:04.688296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 02 08:57:04.688296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 02 08:57:04.688296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 02 08:57:04.688296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application session.commit() Sep 02 08:57:04.688296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 02 08:57:04.688296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 02 08:57:04.688296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 02 08:57:04.689958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._prepare_impl() Sep 02 08:57:04.689958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 02 08:57:04.689958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self.session.flush() Sep 02 08:57:04.689958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 02 08:57:04.689958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._flush(objects) Sep 02 08:57:04.689958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 02 08:57:04.689958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 02 08:57:04.689958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 02 08:57:04.689958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application compat.raise_( Sep 02 08:57:04.689958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 02 08:57:04.689958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception Sep 02 08:57:04.689958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 02 08:57:04.689958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application flush_context.execute() Sep 02 08:57:04.689958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 02 08:57:04.689958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rec.execute(self) Sep 02 08:57:04.689958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 02 08:57:04.689958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 02 08:57:04.690999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 02 08:57:04.690999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 02 08:57:04.690999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 02 08:57:04.690999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 02 08:57:04.690999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 02 08:57:04.690999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 02 08:57:04.690999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 02 08:57:04.690999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 02 08:57:04.690999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 02 08:57:04.690999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 02 08:57:04.690999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 02 08:57:04.690999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 02 08:57:04.690999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 02 08:57:04.690999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 02 08:57:04.690999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 02 08:57:04.690999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception Sep 02 08:57:04.692109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 02 08:57:04.692109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 02 08:57:04.692109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 02 08:57:04.692109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 02 08:57:04.692109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 02 08:57:04.692109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application result = self._query(query) Sep 02 08:57:04.692109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 02 08:57:04.692109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application conn.query(q) Sep 02 08:57:04.692109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 02 08:57:04.692109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 02 08:57:04.692109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 02 08:57:04.692109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application result.read() Sep 02 08:57:04.692109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 02 08:57:04.692109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 02 08:57:04.692109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 02 08:57:04.692109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 02 08:57:04.699429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 02 08:57:04.699429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 02 08:57:04.699429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 02 08:57:04.699429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 02 08:57:04.699429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 02 08:57:04.699429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 02 08:57:04.699429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [parameters: {'user_id': '7b11b125065642fdab68370f3e0ec9c4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 02 08:57:04.699429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 02 08:57:04.699429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:04.699429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:57:04.699429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:04.699429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:04.699429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:04.699429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:04.699429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:04.701112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:04.701112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:04.701112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:04.701112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:04.701112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:04.701112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:04.701112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:04.701112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 02 08:57:04.701112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 02 08:57:04.701112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:04.701112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:04.701112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:57:04.701112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:57:04.701112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:57:04.701112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:57:04.701112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 02 08:57:04.701112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 02 08:57:04.702972 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 02 08:57:04.702972 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 02 08:57:04.702972 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 02 08:57:04.702972 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 02 08:57:04.702972 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 02 08:57:04.702972 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:04.702972 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 504/973] 10.222.0.36 () {60 vars in 1131 bytes} [Fri Sep 2 08:57:04 2022] POST /identity/v3/users => generated 160 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:57:04.702972 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e61eb2-6044-4df4-b435-92fb8f20c282 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:04.750964 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3c5c6-418f-432b-b421-7eface741a26 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:04.753289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:04.788967 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0ad83-a385-454e-ba3e-2db9ad4251e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:04.789575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0ad83-a385-454e-ba3e-2db9ad4251e4 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:04.789850 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0ad83-a385-454e-ba3e-2db9ad4251e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:04.790133 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0ad83-a385-454e-ba3e-2db9ad4251e4 None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:04.791099 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0ad83-a385-454e-ba3e-2db9ad4251e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:04.791233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0ad83-a385-454e-ba3e-2db9ad4251e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:04.792561 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0ad83-a385-454e-ba3e-2db9ad4251e4 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:04.810696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e61eb2-6044-4df4-b435-92fb8f20c282 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:04.811263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 470/974] 10.222.0.36 () {56 vars in 1149 bytes} [Fri Sep 2 08:57:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:04.812277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e03a75-fb9f-41e0-a3da-0f51dc201de6 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:04.812555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e03a75-fb9f-41e0-a3da-0f51dc201de6 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:04.812837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e03a75-fb9f-41e0-a3da-0f51dc201de6 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:04.814800 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 471/975] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:04.819313 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e03a75-fb9f-41e0-a3da-0f51dc201de6 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:04.846497 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e67df94-9e96-48ce-9b91-e65b86865464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:04.847154 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67df94-9e96-48ce-9b91-e65b86865464 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:04.847776 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67df94-9e96-48ce-9b91-e65b86865464 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:04.847776 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e67df94-9e96-48ce-9b91-e65b86865464 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:04.850062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:04.858955 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e67df94-9e96-48ce-9b91-e65b86865464 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:04.861653 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e67df94-9e96-48ce-9b91-e65b86865464 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:04.874414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-5e67df94-9e96-48ce-9b91-e65b86865464 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 08:57:04.874414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:04.874414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:04.874414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:04.874414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:04.874414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:04.874414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:04.874414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:04.874414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:04.874414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:04.874414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:04.874414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:04.874414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:57:04.874414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:57:04.874414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:57:04.874414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:57:04.875419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:04.875419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:04.875419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:57:04.875419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:57:04.875419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:57:04.875419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:57:04.875419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:04.875419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:04.875419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:04.875419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:04.875419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:04.875419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:04.875419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:04.875419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:04.875419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:04.875419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:04.875419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:04.875419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:04.876412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:04.876412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:04.876412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:57:04.876412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:57:04.876412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:57:04.876412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:57:04.876412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:57:04.876412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:57:04.876412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 08:57:04.876412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:04.876412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 505/976] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:57:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:57:04.878774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351fb03-d736-4416-af09-20267d3779eb None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:04.878774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351fb03-d736-4416-af09-20267d3779eb None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:04.878774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351fb03-d736-4416-af09-20267d3779eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:04.896541 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-8351fb03-d736-4416-af09-20267d3779eb None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:04.898031 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8351fb03-d736-4416-af09-20267d3779eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:04.930738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e03a75-fb9f-41e0-a3da-0f51dc201de6 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:04.966989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 506/977] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:04 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:04.967218 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728639b2-5cfd-4ffd-a885-c775613ed8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:04.967796 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728639b2-5cfd-4ffd-a885-c775613ed8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:04.968169 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728639b2-5cfd-4ffd-a885-c775613ed8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:04.968501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728639b2-5cfd-4ffd-a885-c775613ed8c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:04.976058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351fb03-d736-4416-af09-20267d3779eb None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:05.000282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:05.044885 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728639b2-5cfd-4ffd-a885-c775613ed8c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:05.047709 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728639b2-5cfd-4ffd-a885-c775613ed8c0 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:05.112854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8351fb03-d736-4416-af09-20267d3779eb None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:05.139122 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:05.153267 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968a89e-22ca-4fa1-84c3-a10a4a988f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:05.153872 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968a89e-22ca-4fa1-84c3-a10a4a988f58 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:05.154145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968a89e-22ca-4fa1-84c3-a10a4a988f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:05.154412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968a89e-22ca-4fa1-84c3-a10a4a988f58 None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:05.155123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968a89e-22ca-4fa1-84c3-a10a4a988f58 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:05.155431 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968a89e-22ca-4fa1-84c3-a10a4a988f58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:05.157508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968a89e-22ca-4fa1-84c3-a10a4a988f58 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:05.195885 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 472/978] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:57:04 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 08:57:05.204040 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728639b2-5cfd-4ffd-a885-c775613ed8c0 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:05.204363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 507/979] 10.222.0.36 () {56 vars in 1101 bytes} [Fri Sep 2 08:57:04 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:05.248203 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968a89e-22ca-4fa1-84c3-a10a4a988f58 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:05.275464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:05.325854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728639b2-5cfd-4ffd-a885-c775613ed8c0 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:05.375592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48d375-9147-4f7b-8518-3e0e6ae0c52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:05.376286 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48d375-9147-4f7b-8518-3e0e6ae0c52f None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:05.377510 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48d375-9147-4f7b-8518-3e0e6ae0c52f None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:05.377903 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48d375-9147-4f7b-8518-3e0e6ae0c52f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:05.380672 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968a89e-22ca-4fa1-84c3-a10a4a988f58 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:05.390536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48d375-9147-4f7b-8518-3e0e6ae0c52f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:05.392145 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48d375-9147-4f7b-8518-3e0e6ae0c52f None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:05.409308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-fa48d375-9147-4f7b-8518-3e0e6ae0c52f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:57:05.409308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:05.409308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:05.409308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:05.409308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:05.409308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:05.409308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:05.409308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:05.409308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:05.409308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:05.409308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:05.409308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:05.409308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:57:05.409308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:57:05.409308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:57:05.409308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:57:05.410279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:05.410279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:05.410279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:05.410279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:05.410279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:05.410279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:05.410279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:05.410279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:05.410279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:05.410279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:05.410279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:05.410279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:05.410279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:05.410279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:05.410279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:05.410279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:05.410279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:57:05.411205 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:57:05.411205 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:57:05.411205 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:05.411205 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 473/980] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:57:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:05.416706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48d375-9147-4f7b-8518-3e0e6ae0c52f None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:05.430160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c469d017-7540-4adf-9357-4dd817179bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:05.430786 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469d017-7540-4adf-9357-4dd817179bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:05.431062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469d017-7540-4adf-9357-4dd817179bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:05.431392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469d017-7540-4adf-9357-4dd817179bb0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:05.431983 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c469d017-7540-4adf-9357-4dd817179bb0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:05.434248 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c469d017-7540-4adf-9357-4dd817179bb0 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:05.449575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-c469d017-7540-4adf-9357-4dd817179bb0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 08:57:05.449575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:05.449575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:57:05.449575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:57:05.449575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:57:05.449575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:57:05.449575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:57:05.449575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:57:05.449575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 08:57:05.449575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:05.449575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:57:05.449575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:05.449575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:05.449575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:05.449575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:05.449575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:05.449575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:05.450638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:05.450638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:05.450638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:05.450638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:05.450638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:05.450638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:05.450638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:57:05.450638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:57:05.450638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:57:05.450638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:57:05.450638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:05.450638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:05.450638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:05.450638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:05.450638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:05.450638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:05.451664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:05.451664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:05.451664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:05.451664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:05.451664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:05.451664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:05.451664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:05.451664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:05.451664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:05.451664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:05.451664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:57:05.451664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:57:05.451664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 08:57:05.451664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:05.451664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 508/981] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:57:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:05.457703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c469d017-7540-4adf-9357-4dd817179bb0 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:05.508421 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:05.532347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48d375-9147-4f7b-8518-3e0e6ae0c52f None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:05.548084 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:05.566928 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c469d017-7540-4adf-9357-4dd817179bb0 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:05.572290 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9ec4d-dbfd-4a62-bbc6-8a1f40446b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:05.573136 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9ec4d-dbfd-4a62-bbc6-8a1f40446b9d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:05.573611 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9ec4d-dbfd-4a62-bbc6-8a1f40446b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:05.574080 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e9ec4d-dbfd-4a62-bbc6-8a1f40446b9d None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:05.574730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9ec4d-dbfd-4a62-bbc6-8a1f40446b9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:05.575034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9ec4d-dbfd-4a62-bbc6-8a1f40446b9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:05.576482 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e9ec4d-dbfd-4a62-bbc6-8a1f40446b9d None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:05.593586 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 474/982] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:57:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:05.594419 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9ec4d-dbfd-4a62-bbc6-8a1f40446b9d None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:05.603260 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70eecf3-f11b-4f27-ac5b-c73d65e58395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:05.603869 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70eecf3-f11b-4f27-ac5b-c73d65e58395 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:05.604147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70eecf3-f11b-4f27-ac5b-c73d65e58395 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:05.604421 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70eecf3-f11b-4f27-ac5b-c73d65e58395 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:05.605049 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70eecf3-f11b-4f27-ac5b-c73d65e58395 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:05.605349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70eecf3-f11b-4f27-ac5b-c73d65e58395 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:05.606712 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70eecf3-f11b-4f27-ac5b-c73d65e58395 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:05.611647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 509/983] 10.222.0.36 () {56 vars in 1112 bytes} [Fri Sep 2 08:57:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:05.617068 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70eecf3-f11b-4f27-ac5b-c73d65e58395 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:05.706545 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e9ec4d-dbfd-4a62-bbc6-8a1f40446b9d None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:05.728346 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70eecf3-f11b-4f27-ac5b-c73d65e58395 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:05.739541 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:05.743591 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd851cd-b222-413f-a503-3b44d5220afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:05.744255 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd851cd-b222-413f-a503-3b44d5220afc None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:05.744534 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd851cd-b222-413f-a503-3b44d5220afc None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:05.744803 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd851cd-b222-413f-a503-3b44d5220afc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:05.752277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd851cd-b222-413f-a503-3b44d5220afc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:05.753706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd851cd-b222-413f-a503-3b44d5220afc None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:05.765196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-ecd851cd-b222-413f-a503-3b44d5220afc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:57:05.765196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:05.765196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:05.765196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:05.765196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:05.765196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:05.765196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:05.765196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:05.765196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:05.765196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:05.765196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:05.765196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:05.765196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:57:05.765196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:57:05.765196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:57:05.766124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:57:05.766124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:05.766124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:05.766124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:05.766124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:05.766124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:05.766124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:05.766124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:05.766124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:05.766124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:05.766124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:05.766124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:05.766124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:05.766124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:05.766124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:05.766124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:05.766124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:05.767621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:57:05.767621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:57:05.767621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:57:05.767621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:57:05.767621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:57:05.767621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:57:05.767621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:57:05.767621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:05.768068 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67e2d4-e4c0-43ba-8ce3-f601b7ac15f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:05.768068 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67e2d4-e4c0-43ba-8ce3-f601b7ac15f9 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:05.768068 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67e2d4-e4c0-43ba-8ce3-f601b7ac15f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:05.768300 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 475/984] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:57:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:57:05.768378 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67e2d4-e4c0-43ba-8ce3-f601b7ac15f9 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:05.768378 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67e2d4-e4c0-43ba-8ce3-f601b7ac15f9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:05.769024 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67e2d4-e4c0-43ba-8ce3-f601b7ac15f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:05.770005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67e2d4-e4c0-43ba-8ce3-f601b7ac15f9 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:05.772488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd851cd-b222-413f-a503-3b44d5220afc None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:05.780689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 510/985] 10.222.0.36 () {56 vars in 1112 bytes} [Fri Sep 2 08:57:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:05.780836 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67e2d4-e4c0-43ba-8ce3-f601b7ac15f9 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:05.813620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:05.884137 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd851cd-b222-413f-a503-3b44d5220afc None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:05.887490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67e2d4-e4c0-43ba-8ce3-f601b7ac15f9 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:05.919538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8a147-fafd-48de-b1b8-c6250b171b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:05.920140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8a147-fafd-48de-b1b8-c6250b171b51 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:05.920418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8a147-fafd-48de-b1b8-c6250b171b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:05.920685 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af8a147-fafd-48de-b1b8-c6250b171b51 None admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:05.921369 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8a147-fafd-48de-b1b8-c6250b171b51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:05.921669 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8a147-fafd-48de-b1b8-c6250b171b51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:05.923098 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37dbd18-1378-4df8-85f9-77d57233844a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:05.923241 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af8a147-fafd-48de-b1b8-c6250b171b51 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:05.923752 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37dbd18-1378-4df8-85f9-77d57233844a None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:05.924037 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37dbd18-1378-4df8-85f9-77d57233844a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:05.924312 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37dbd18-1378-4df8-85f9-77d57233844a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:05.925008 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37dbd18-1378-4df8-85f9-77d57233844a None admin] RBAC: Adding query filter params (interface=public, service_id=8f648dacb15f4b2d9f2b8575700253a4, region_id=RegionOne) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:05.925322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37dbd18-1378-4df8-85f9-77d57233844a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:05.926686 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37dbd18-1378-4df8-85f9-77d57233844a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:05.932155 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 511/986] 10.222.0.36 () {56 vars in 1251 bytes} [Fri Sep 2 08:57:05 2022] GET /identity/v3/endpoints?service_id=8f648dacb15f4b2d9f2b8575700253a4&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:05.936346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 476/987] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:57:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:05.937810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37dbd18-1378-4df8-85f9-77d57233844a None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:05.941807 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8a147-fafd-48de-b1b8-c6250b171b51 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:05.989109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:06.037870 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:06.047326 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af8a147-fafd-48de-b1b8-c6250b171b51 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:06.056370 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37dbd18-1378-4df8-85f9-77d57233844a None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:06.081563 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78823adf-ebb8-486f-819b-6e87472fc8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:06.082164 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78823adf-ebb8-486f-819b-6e87472fc8be None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:06.082446 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78823adf-ebb8-486f-819b-6e87472fc8be None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:06.082734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78823adf-ebb8-486f-819b-6e87472fc8be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:06.083478 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78823adf-ebb8-486f-819b-6e87472fc8be None admin] RBAC: Adding query filter params (role.id=a604599bb8af4abfaada3a92ac76b1bd, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=76ebecce283f491e82a961d7f739878f) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:06.083789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78823adf-ebb8-486f-819b-6e87472fc8be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:06.085213 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78823adf-ebb8-486f-819b-6e87472fc8be None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:06.090766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 477/988] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:57:05 2022] GET /identity/v3/role_assignments?user.id=76ebecce283f491e82a961d7f739878f&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=a604599bb8af4abfaada3a92ac76b1bd => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:06.091056 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2589e66-246b-4023-b6dd-de5203efb094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:06.091681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2589e66-246b-4023-b6dd-de5203efb094 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:06.092020 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2589e66-246b-4023-b6dd-de5203efb094 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:06.092295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2589e66-246b-4023-b6dd-de5203efb094 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:06.092890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2589e66-246b-4023-b6dd-de5203efb094 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:06.094249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2589e66-246b-4023-b6dd-de5203efb094 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:06.096473 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dffe9-8388-4c3e-985b-349adceebfcb None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:06.096757 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dffe9-8388-4c3e-985b-349adceebfcb None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:06.097078 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dffe9-8388-4c3e-985b-349adceebfcb None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:06.098460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 478/989] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:06.103116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 512/990] 10.222.0.36 () {56 vars in 1087 bytes} [Fri Sep 2 08:57:05 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 08:57:06.104312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4da74a-3848-494c-9fa3-fe77ff00ee52 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:06.104594 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4da74a-3848-494c-9fa3-fe77ff00ee52 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:06.104937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4da74a-3848-494c-9fa3-fe77ff00ee52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:06.133187 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-5a4da74a-3848-494c-9fa3-fe77ff00ee52 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:06.134428 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-5a4da74a-3848-494c-9fa3-fe77ff00ee52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:06.218284 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 479/991] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:06 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:06.483349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2589e66-246b-4023-b6dd-de5203efb094 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:06.589738 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2589e66-246b-4023-b6dd-de5203efb094 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:06.591538 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:06.625675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38705bfb-4f1a-4544-9ab2-da5b9d36d1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:06.626233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38705bfb-4f1a-4544-9ab2-da5b9d36d1be None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:06.627143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38705bfb-4f1a-4544-9ab2-da5b9d36d1be None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:06.627422 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38705bfb-4f1a-4544-9ab2-da5b9d36d1be None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:06.628010 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38705bfb-4f1a-4544-9ab2-da5b9d36d1be None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:06.629372 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38705bfb-4f1a-4544-9ab2-da5b9d36d1be None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:06.638085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-38705bfb-4f1a-4544-9ab2-da5b9d36d1be None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 08:57:06.638085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:06.638085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:57:06.638085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:57:06.638085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:57:06.638085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:57:06.638085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:57:06.638085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:57:06.638085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 08:57:06.638085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:06.638085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:57:06.638085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:06.638085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:06.638085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:06.638085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:06.638085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:06.638085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:06.639060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:06.639060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:06.639060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:06.639060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:06.639060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:06.639060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:06.639060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:57:06.639060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:57:06.639060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:57:06.639060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:57:06.639060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:06.639060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:06.639060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:06.639060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:06.639060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:06.639060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:06.639962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:06.639962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:06.639962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:06.639962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:06.639962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:06.639962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:06.639962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:06.639962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:06.639962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:06.639962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:06.639962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:57:06.639962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:57:06.639962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 08:57:06.639962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:06.639962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 513/992] 10.222.0.36 () {56 vars in 1126 bytes} [Fri Sep 2 08:57:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:06.645082 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4da74a-3848-494c-9fa3-fe77ff00ee52 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:06.731367 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:06.749035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4da74a-3848-494c-9fa3-fe77ff00ee52 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:06.783183 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff306d-3fc2-4817-954e-4a03172f520f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:06.783794 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff306d-3fc2-4817-954e-4a03172f520f None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:06.784073 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff306d-3fc2-4817-954e-4a03172f520f None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:06.784340 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff306d-3fc2-4817-954e-4a03172f520f None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:06.784977 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff306d-3fc2-4817-954e-4a03172f520f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:06.785280 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff306d-3fc2-4817-954e-4a03172f520f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:06.786634 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff306d-3fc2-4817-954e-4a03172f520f None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:06.796314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 480/993] 10.222.0.36 () {56 vars in 1122 bytes} [Fri Sep 2 08:57:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:06.797147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38705bfb-4f1a-4544-9ab2-da5b9d36d1be None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:06.896798 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38705bfb-4f1a-4544-9ab2-da5b9d36d1be None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:06.930043 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a30c47-2325-4a8b-9369-e69bfa81ae2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:06.930664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a30c47-2325-4a8b-9369-e69bfa81ae2d None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:06.930942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a30c47-2325-4a8b-9369-e69bfa81ae2d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:06.931243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a30c47-2325-4a8b-9369-e69bfa81ae2d None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:06.931862 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a30c47-2325-4a8b-9369-e69bfa81ae2d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:06.932161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a30c47-2325-4a8b-9369-e69bfa81ae2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:06.933781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a30c47-2325-4a8b-9369-e69bfa81ae2d None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:06.938498 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 514/994] 10.222.0.36 () {56 vars in 1122 bytes} [Fri Sep 2 08:57:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:06.944449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff306d-3fc2-4817-954e-4a03172f520f None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:07.040474 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff306d-3fc2-4817-954e-4a03172f520f None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:07.073964 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b4f95d-c8e3-4a87-9f96-b7afbba6eb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:07.074571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4f95d-c8e3-4a87-9f96-b7afbba6eb10 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:07.074855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4f95d-c8e3-4a87-9f96-b7afbba6eb10 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:07.075157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b4f95d-c8e3-4a87-9f96-b7afbba6eb10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:07.075866 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b4f95d-c8e3-4a87-9f96-b7afbba6eb10 None admin] RBAC: Adding query filter params (interface=admin, service_id=ca24d292b70d40ababa408b83d3e59f9, region_id=RegionOne) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:07.076179 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b4f95d-c8e3-4a87-9f96-b7afbba6eb10 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:07.077530 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b4f95d-c8e3-4a87-9f96-b7afbba6eb10 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:07.087285 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 481/995] 10.222.0.36 () {56 vars in 1249 bytes} [Fri Sep 2 08:57:06 2022] GET /identity/v3/endpoints?service_id=ca24d292b70d40ababa408b83d3e59f9&interface=admin®ion_id=RegionOne => generated 193 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:07.089006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a30c47-2325-4a8b-9369-e69bfa81ae2d None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:07.128021 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:07.209003 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a30c47-2325-4a8b-9369-e69bfa81ae2d None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:07.245205 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb7eb01-9cae-4c3a-92eb-c5c871ba4077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:07.245815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7eb01-9cae-4c3a-92eb-c5c871ba4077 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:07.246147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7eb01-9cae-4c3a-92eb-c5c871ba4077 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:07.246422 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb7eb01-9cae-4c3a-92eb-c5c871ba4077 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:07.247020 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb7eb01-9cae-4c3a-92eb-c5c871ba4077 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:07.248420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb7eb01-9cae-4c3a-92eb-c5c871ba4077 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:07.263173 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 515/996] 10.222.0.36 () {56 vars in 1087 bytes} [Fri Sep 2 08:57:07 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 08:57:08.003865 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f6b0a-c1e6-4d50-9166-b9df179ac712 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:08.004397 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f6b0a-c1e6-4d50-9166-b9df179ac712 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:08.004397 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f6b0a-c1e6-4d50-9166-b9df179ac712 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:08.005772 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 482/997] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:08.010750 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057c5b6-effc-4483-a99f-e9772553adbf None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:08.011191 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057c5b6-effc-4483-a99f-e9772553adbf None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:08.011418 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057c5b6-effc-4483-a99f-e9772553adbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:08.029484 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-1057c5b6-effc-4483-a99f-e9772553adbf None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:08.030669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-1057c5b6-effc-4483-a99f-e9772553adbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:08.078497 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820306b-36c2-4c4c-8351-d06dc6720e28 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:08.078782 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820306b-36c2-4c4c-8351-d06dc6720e28 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:08.079059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5820306b-36c2-4c4c-8351-d06dc6720e28 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:08.080652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 483/998] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:57:08.085405 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f5e56-b68a-4b45-bf19-88bb7b1e775d None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:08.085692 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f5e56-b68a-4b45-bf19-88bb7b1e775d None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:08.085967 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f5e56-b68a-4b45-bf19-88bb7b1e775d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:08.103139 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 516/999] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:08 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:08.104473 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-3a4f5e56-b68a-4b45-bf19-88bb7b1e775d None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:08.105678 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-3a4f5e56-b68a-4b45-bf19-88bb7b1e775d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:08.118845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:08.178490 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 484/1000] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:08 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 08:57:08.372689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057c5b6-effc-4483-a99f-e9772553adbf None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:08.397626 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:08.450608 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f5e56-b68a-4b45-bf19-88bb7b1e775d None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:08.475061 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057c5b6-effc-4483-a99f-e9772553adbf None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:08.509008 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5aea1-47e8-4110-96f2-54acff501591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:08.509636 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5aea1-47e8-4110-96f2-54acff501591 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:08.509914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5aea1-47e8-4110-96f2-54acff501591 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:08.510187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5aea1-47e8-4110-96f2-54acff501591 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:08.510766 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5aea1-47e8-4110-96f2-54acff501591 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:08.512161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5aea1-47e8-4110-96f2-54acff501591 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:08.520802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-2ad5aea1-47e8-4110-96f2-54acff501591 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 08:57:08.520802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:08.520802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:57:08.520802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:57:08.520802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:57:08.520802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:57:08.520802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:57:08.520802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:57:08.520802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 08:57:08.520802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:08.520802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:57:08.520802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:08.520802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:08.520802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:08.520802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:08.520802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:08.520802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:08.521802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:08.521802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:08.521802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:08.521802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:08.521802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:08.521802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:08.521802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:57:08.521802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:57:08.521802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:57:08.521802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:57:08.521802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:08.521802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:08.521802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:08.521802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:08.521802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:08.521802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:08.523131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:08.523131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:08.523131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:08.523131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:08.523131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:08.523131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:08.523131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:08.523131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:08.523131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:08.523131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:08.523131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:57:08.523131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:57:08.523131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 08:57:08.523131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:08.523131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 517/1001] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:57:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:08.527577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5aea1-47e8-4110-96f2-54acff501591 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:08.557231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4f5e56-b68a-4b45-bf19-88bb7b1e775d None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:08.593472 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed46b6-08e9-405d-891c-d314b5a90fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:08.594421 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed46b6-08e9-405d-891c-d314b5a90fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:08.594851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed46b6-08e9-405d-891c-d314b5a90fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:08.595853 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed46b6-08e9-405d-891c-d314b5a90fa0 None admin] PATH_INFO: `/v3/services/image` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:08.596645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ed46b6-08e9-405d-891c-d314b5a90fa0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:08.598257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ed46b6-08e9-405d-891c-d314b5a90fa0 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:08.607784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-96ed46b6-08e9-405d-891c-d314b5a90fa0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 08:57:08.607784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:08.607784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:57:08.607784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:57:08.607784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:57:08.607784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:57:08.607784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:57:08.607784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:57:08.607784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 08:57:08.607784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:08.607784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:57:08.607784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:08.607784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:08.607784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:08.607784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:08.607784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:08.607784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:08.608971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:08.608971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:08.608971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:08.608971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:08.608971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:08.608971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:08.608971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:57:08.608971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:57:08.608971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:57:08.608971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:57:08.608971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:08.608971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:08.608971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:08.608971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:08.608971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:08.608971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:08.610059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:08.610059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:08.610059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:08.610059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:08.610059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:08.610059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:08.610059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:08.610059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:08.610059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:08.610059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:08.610059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:57:08.610059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:57:08.610059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 08:57:08.610059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:08.612304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 485/1002] 10.222.0.36 () {56 vars in 1105 bytes} [Fri Sep 2 08:57:08 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:57:08.615764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:08.618419 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed46b6-08e9-405d-891c-d314b5a90fa0 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:08.629437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:08.629631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5aea1-47e8-4110-96f2-54acff501591 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:08.666156 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b4a1e9-a87d-4ced-a93a-895729ec0bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:08.666767 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4a1e9-a87d-4ced-a93a-895729ec0bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:08.667047 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4a1e9-a87d-4ced-a93a-895729ec0bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:08.667348 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b4a1e9-a87d-4ced-a93a-895729ec0bd3 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:08.667968 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b4a1e9-a87d-4ced-a93a-895729ec0bd3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:08.668273 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b4a1e9-a87d-4ced-a93a-895729ec0bd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:08.669646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b4a1e9-a87d-4ced-a93a-895729ec0bd3 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:08.675341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 518/1003] 10.222.0.36 () {56 vars in 1112 bytes} [Fri Sep 2 08:57:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:08.681046 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b4a1e9-a87d-4ced-a93a-895729ec0bd3 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:08.728799 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed46b6-08e9-405d-891c-d314b5a90fa0 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:08.765418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8edb59-1749-4d59-9f71-a4ce03d114c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:08.766040 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8edb59-1749-4d59-9f71-a4ce03d114c0 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:08.766314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8edb59-1749-4d59-9f71-a4ce03d114c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:08.766583 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8edb59-1749-4d59-9f71-a4ce03d114c0 None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:08.767274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8edb59-1749-4d59-9f71-a4ce03d114c0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:08.767581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8edb59-1749-4d59-9f71-a4ce03d114c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:08.768925 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8edb59-1749-4d59-9f71-a4ce03d114c0 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:08.775109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 486/1004] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:57:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:08.779541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8edb59-1749-4d59-9f71-a4ce03d114c0 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:08.793037 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b4a1e9-a87d-4ced-a93a-895729ec0bd3 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:08.830440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658433d-325a-4d18-b914-6f5d9fd48a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:08.831042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658433d-325a-4d18-b914-6f5d9fd48a15 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:08.831345 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658433d-325a-4d18-b914-6f5d9fd48a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:08.831742 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658433d-325a-4d18-b914-6f5d9fd48a15 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:08.832370 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658433d-325a-4d18-b914-6f5d9fd48a15 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:08.832666 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658433d-325a-4d18-b914-6f5d9fd48a15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:08.834033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658433d-325a-4d18-b914-6f5d9fd48a15 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:08.843645 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 519/1005] 10.222.0.36 () {56 vars in 1112 bytes} [Fri Sep 2 08:57:08 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:08.844437 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658433d-325a-4d18-b914-6f5d9fd48a15 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:08.852708 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:08.864729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:08.884652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8edb59-1749-4d59-9f71-a4ce03d114c0 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:08.918411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a84b66-a95c-453c-8df9-253de6918d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:08.919024 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a84b66-a95c-453c-8df9-253de6918d5a None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:08.919339 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a84b66-a95c-453c-8df9-253de6918d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:08.919601 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a84b66-a95c-453c-8df9-253de6918d5a None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:08.920229 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a84b66-a95c-453c-8df9-253de6918d5a None admin] RBAC: Adding query filter params (type=image) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:08.920531 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a84b66-a95c-453c-8df9-253de6918d5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:08.921885 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a84b66-a95c-453c-8df9-253de6918d5a None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:08.926757 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 487/1006] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:57:08 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:08.964511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658433d-325a-4d18-b914-6f5d9fd48a15 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:09.002127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0058be1-1b26-4aaa-8e09-973ee79c2dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:09.002749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0058be1-1b26-4aaa-8e09-973ee79c2dad None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:09.003023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0058be1-1b26-4aaa-8e09-973ee79c2dad None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:09.003328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0058be1-1b26-4aaa-8e09-973ee79c2dad None admin] PATH_INFO: `/v3/endpoints` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:09.004007 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0058be1-1b26-4aaa-8e09-973ee79c2dad None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:09.005417 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0058be1-1b26-4aaa-8e09-973ee79c2dad None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:09.027019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 520/1007] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 08:57:08 2022] POST /identity/v3/endpoints => generated 334 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:57:09.146209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab82277-8a91-4fe1-a396-5d7dbee5b563 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:09.146726 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab82277-8a91-4fe1-a396-5d7dbee5b563 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:09.146726 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab82277-8a91-4fe1-a396-5d7dbee5b563 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:09.148579 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 488/1008] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:09.154399 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb6efb-783f-4617-9ef8-01da858b7c2a None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:09.154903 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb6efb-783f-4617-9ef8-01da858b7c2a None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:09.155235 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb6efb-783f-4617-9ef8-01da858b7c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:09.177061 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-d2cb6efb-783f-4617-9ef8-01da858b7c2a None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:09.178251 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-d2cb6efb-783f-4617-9ef8-01da858b7c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:09.249638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:09.258059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 521/1009] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:09 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:09.530915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab82277-8a91-4fe1-a396-5d7dbee5b563 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:09.633130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab82277-8a91-4fe1-a396-5d7dbee5b563 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:09.667371 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f0746-e035-4d7f-83a9-3f448474d8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:09.668000 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f0746-e035-4d7f-83a9-3f448474d8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:09.668283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f0746-e035-4d7f-83a9-3f448474d8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:09.668554 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828f0746-e035-4d7f-83a9-3f448474d8b5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:09.669137 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f0746-e035-4d7f-83a9-3f448474d8b5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:09.670502 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828f0746-e035-4d7f-83a9-3f448474d8b5 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:09.673406 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:09.680779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-828f0746-e035-4d7f-83a9-3f448474d8b5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 08:57:09.680779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:09.680779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:57:09.680779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:57:09.680779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:57:09.680779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:57:09.680779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:57:09.680779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:57:09.680779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 08:57:09.680779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:09.680779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:57:09.680779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:09.680779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:09.680779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:09.680779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:09.680779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:09.680779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:09.681754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:09.681754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:09.681754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:09.681754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:09.681754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:09.681754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:09.681754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:57:09.681754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:57:09.681754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:57:09.681754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:57:09.681754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:09.681754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:09.681754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:09.681754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:09.681754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:09.681754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:09.682610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:09.682610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:09.682610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:09.682610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:09.682610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:09.682610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:09.682610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:09.682610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:09.682610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:09.682610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:09.682610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:57:09.682610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:57:09.682610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 08:57:09.682610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:09.691060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 489/1010] 10.222.0.36 () {56 vars in 1126 bytes} [Fri Sep 2 08:57:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:09.692181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb6efb-783f-4617-9ef8-01da858b7c2a None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:09.793802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb6efb-783f-4617-9ef8-01da858b7c2a None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:09.828033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234ff90-db8f-4856-92cf-d8e981e3cd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:09.828646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234ff90-db8f-4856-92cf-d8e981e3cd20 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:09.828927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234ff90-db8f-4856-92cf-d8e981e3cd20 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:09.829198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e234ff90-db8f-4856-92cf-d8e981e3cd20 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:09.830381 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234ff90-db8f-4856-92cf-d8e981e3cd20 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:09.830685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234ff90-db8f-4856-92cf-d8e981e3cd20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:09.832077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e234ff90-db8f-4856-92cf-d8e981e3cd20 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:09.836915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 522/1011] 10.222.0.36 () {56 vars in 1122 bytes} [Fri Sep 2 08:57:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:09.842564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f0746-e035-4d7f-83a9-3f448474d8b5 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:09.941191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828f0746-e035-4d7f-83a9-3f448474d8b5 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:09.974176 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ef470-4e34-415e-8e9a-ef0656868c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:09.974775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ef470-4e34-415e-8e9a-ef0656868c90 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:09.975049 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ef470-4e34-415e-8e9a-ef0656868c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:09.975342 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ef470-4e34-415e-8e9a-ef0656868c90 None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:09.975957 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ef470-4e34-415e-8e9a-ef0656868c90 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:09.976257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ef470-4e34-415e-8e9a-ef0656868c90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:09.977596 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ef470-4e34-415e-8e9a-ef0656868c90 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:09.982447 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 490/1012] 10.222.0.36 () {56 vars in 1122 bytes} [Fri Sep 2 08:57:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:09.988015 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234ff90-db8f-4856-92cf-d8e981e3cd20 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:10.053829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:10.093473 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e234ff90-db8f-4856-92cf-d8e981e3cd20 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:10.128109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97e6e3-1333-4d76-b74a-55972226955c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:10.128714 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97e6e3-1333-4d76-b74a-55972226955c None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:10.128994 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97e6e3-1333-4d76-b74a-55972226955c None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:10.129264 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97e6e3-1333-4d76-b74a-55972226955c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:10.129932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b97e6e3-1333-4d76-b74a-55972226955c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:10.131323 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b97e6e3-1333-4d76-b74a-55972226955c None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:10.152081 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 523/1013] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 08:57:09 2022] POST /identity/v3/endpoints => generated 325 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:57:10.857901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04ec45-643c-443e-890d-2d9cd5fb4943 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:10.858410 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04ec45-643c-443e-890d-2d9cd5fb4943 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:10.858505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea04ec45-643c-443e-890d-2d9cd5fb4943 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:10.859978 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 491/1014] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:10.865053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea7790-81f5-4aee-8a00-9f3aea108629 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:10.865345 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea7790-81f5-4aee-8a00-9f3aea108629 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:10.865625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea7790-81f5-4aee-8a00-9f3aea108629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:10.883440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-e2ea7790-81f5-4aee-8a00-9f3aea108629 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:10.884616 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-e2ea7790-81f5-4aee-8a00-9f3aea108629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:10.960559 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 524/1015] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:10 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:11.072722 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a4564-7760-4311-8b5b-2431e466f4df None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:11.073033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a4564-7760-4311-8b5b-2431e466f4df None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:11.073274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a4564-7760-4311-8b5b-2431e466f4df None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:11.074749 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 492/1016] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:11.080236 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475adda-3aab-471b-bbed-c1fadf29fa36 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:11.080712 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475adda-3aab-471b-bbed-c1fadf29fa36 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:11.080893 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475adda-3aab-471b-bbed-c1fadf29fa36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:11.099709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-1475adda-3aab-471b-bbed-c1fadf29fa36 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:11.100870 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-1475adda-3aab-471b-bbed-c1fadf29fa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:11.144795 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:11.171254 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 525/1017] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:11 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:11.239004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a4564-7760-4311-8b5b-2431e466f4df None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:11.305937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:11.350405 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a4564-7760-4311-8b5b-2431e466f4df None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:11.387462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915119bc-b53c-47f8-839f-2605d952756b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:11.388373 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915119bc-b53c-47f8-839f-2605d952756b None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:11.388830 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915119bc-b53c-47f8-839f-2605d952756b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:11.389277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915119bc-b53c-47f8-839f-2605d952756b None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:11.390095 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915119bc-b53c-47f8-839f-2605d952756b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:11.391693 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915119bc-b53c-47f8-839f-2605d952756b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:11.409560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 493/1018] 10.222.0.36 () {60 vars in 1141 bytes} [Fri Sep 2 08:57:11 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:57:11.449102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475adda-3aab-471b-bbed-c1fadf29fa36 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:11.562611 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475adda-3aab-471b-bbed-c1fadf29fa36 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:11.608385 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521ac8b-8106-4a48-a414-60296f86a34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:11.609404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521ac8b-8106-4a48-a414-60296f86a34d None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:11.609902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521ac8b-8106-4a48-a414-60296f86a34d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:11.610383 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6521ac8b-8106-4a48-a414-60296f86a34d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:11.620798 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6521ac8b-8106-4a48-a414-60296f86a34d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:57:11.625886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-6521ac8b-8106-4a48-a414-60296f86a34d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 08:57:11.625886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:11.625886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:11.625886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:11.625886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:11.625886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:11.625886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:11.625886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:11.625886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:11.625886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:11.625886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:11.625886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:11.625886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:57:11.625886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:57:11.625886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:57:11.625886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:57:11.626892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:57:11.626892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:57:11.626892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:57:11.626892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:57:11.626892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:11.626892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:11.626892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:11.626892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:11.626892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:11.626892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:11.626892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:11.626892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:11.626892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:11.626892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:11.626892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:11.626892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:11.627810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:11.627810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:11.627810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:11.627810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:11.627810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:57:11.627810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:57:11.627810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:57:11.627810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:57:11.627810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:57:11.627810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:57:11.627810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 08:57:11.627810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:11.628484 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 526/1019] 10.222.0.36 () {56 vars in 1120 bytes} [Fri Sep 2 08:57:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:11.635034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915119bc-b53c-47f8-839f-2605d952756b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:11.740913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915119bc-b53c-47f8-839f-2605d952756b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:11.764427 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:11.777649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6602c-beff-4480-8203-e21334b67567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:11.778256 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6602c-beff-4480-8203-e21334b67567 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:11.778529 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6602c-beff-4480-8203-e21334b67567 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:11.778791 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b6602c-beff-4480-8203-e21334b67567 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:11.779539 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6602c-beff-4480-8203-e21334b67567 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:11.779843 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6602c-beff-4480-8203-e21334b67567 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:11.781461 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b6602c-beff-4480-8203-e21334b67567 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:11.791462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 494/1020] 10.222.0.36 () {56 vars in 1115 bytes} [Fri Sep 2 08:57:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:11.796817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521ac8b-8106-4a48-a414-60296f86a34d None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:11.832981 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:11.901312 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6521ac8b-8106-4a48-a414-60296f86a34d None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:11.936622 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4838cf9-572b-4f39-b00d-45246e2518fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:11.937263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4838cf9-572b-4f39-b00d-45246e2518fb None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:11.937537 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4838cf9-572b-4f39-b00d-45246e2518fb None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:11.937800 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4838cf9-572b-4f39-b00d-45246e2518fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:11.948746 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4838cf9-572b-4f39-b00d-45246e2518fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:11.950190 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4838cf9-572b-4f39-b00d-45246e2518fb None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:11.965873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-a4838cf9-572b-4f39-b00d-45246e2518fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:57:11.965873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:11.965873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:11.965873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:11.965873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:11.965873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:11.965873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:11.965873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:11.965873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:11.965873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:11.965873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:11.965873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:11.965873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:57:11.965873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:57:11.965873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:57:11.965873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:57:11.966867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:11.966867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:11.966867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:11.966867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:11.966867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:11.966867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:11.966867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:11.966867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:11.966867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:11.966867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:11.966867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:11.966867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:11.966867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:11.966867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:11.966867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:11.966867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:11.966867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:57:11.967836 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:57:11.967836 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:57:11.967836 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:11.971391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 527/1021] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:57:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:11.972765 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6602c-beff-4480-8203-e21334b67567 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:12.074294 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b6602c-beff-4480-8203-e21334b67567 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:12.096670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4da330-4a2b-49aa-a74f-e7e63275d9a5 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:12.097209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4da330-4a2b-49aa-a74f-e7e63275d9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:12.097209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4da330-4a2b-49aa-a74f-e7e63275d9a5 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:12.098697 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 528/1022] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:12.103637 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a4fff-8101-40d0-b89d-ad5a5a675d14 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:12.103921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a4fff-8101-40d0-b89d-ad5a5a675d14 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:12.104198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a4fff-8101-40d0-b89d-ad5a5a675d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:12.110100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11f421-ae6d-48d2-8fb2-80eb5c3397b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:12.110709 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11f421-ae6d-48d2-8fb2-80eb5c3397b6 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:12.110983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11f421-ae6d-48d2-8fb2-80eb5c3397b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:12.111286 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11f421-ae6d-48d2-8fb2-80eb5c3397b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:12.112011 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11f421-ae6d-48d2-8fb2-80eb5c3397b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:12.112313 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11f421-ae6d-48d2-8fb2-80eb5c3397b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:12.113648 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11f421-ae6d-48d2-8fb2-80eb5c3397b6 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:12.122685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-c12a4fff-8101-40d0-b89d-ad5a5a675d14 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:12.123913 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c12a4fff-8101-40d0-b89d-ad5a5a675d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:12.131130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 495/1023] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:57:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:12.132103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11f421-ae6d-48d2-8fb2-80eb5c3397b6 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:12.182981 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:12.202636 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 529/1024] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 08:57:12.243490 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11f421-ae6d-48d2-8fb2-80eb5c3397b6 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:12.278434 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd214a2-5a47-41cf-9ec4-c3637bd9d18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:12.279062 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd214a2-5a47-41cf-9ec4-c3637bd9d18c None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:12.279365 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd214a2-5a47-41cf-9ec4-c3637bd9d18c None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:12.279637 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd214a2-5a47-41cf-9ec4-c3637bd9d18c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:12.287341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd214a2-5a47-41cf-9ec4-c3637bd9d18c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:12.288824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd214a2-5a47-41cf-9ec4-c3637bd9d18c None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:12.301052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-1fd214a2-5a47-41cf-9ec4-c3637bd9d18c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 08:57:12.301052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:12.301052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:12.301052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:12.301052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:12.301052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:12.301052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:12.301052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:12.301052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:12.301052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:12.301052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:12.301052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:12.301052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:57:12.301052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:57:12.301052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:57:12.301052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:57:12.302025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:12.302025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:12.302025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:57:12.302025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:57:12.302025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:57:12.302025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:57:12.302025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:12.302025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:12.302025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:12.302025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:12.302025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:12.302025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:12.302025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:12.302025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:12.302025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:12.302025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:12.302025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:12.302025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:12.302966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:12.302966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:12.302966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:57:12.302966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:57:12.302966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:57:12.302966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:57:12.302966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:57:12.302966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:57:12.302966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 08:57:12.302966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:12.306792 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 496/1025] 10.222.0.36 () {56 vars in 1093 bytes} [Fri Sep 2 08:57:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:57:12.307868 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a4fff-8101-40d0-b89d-ad5a5a675d14 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:12.327596 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:12.414294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a4fff-8101-40d0-b89d-ad5a5a675d14 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:12.449155 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30284d5b-59b5-412f-84ce-ea85c8858d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:12.449758 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30284d5b-59b5-412f-84ce-ea85c8858d88 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:12.450036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30284d5b-59b5-412f-84ce-ea85c8858d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:12.450305 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30284d5b-59b5-412f-84ce-ea85c8858d88 None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:12.451022 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30284d5b-59b5-412f-84ce-ea85c8858d88 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:12.451352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30284d5b-59b5-412f-84ce-ea85c8858d88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:12.452775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30284d5b-59b5-412f-84ce-ea85c8858d88 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:12.470653 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 530/1026] 10.222.0.36 () {56 vars in 1133 bytes} [Fri Sep 2 08:57:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:12.476090 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30284d5b-59b5-412f-84ce-ea85c8858d88 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:12.476216 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd214a2-5a47-41cf-9ec4-c3637bd9d18c None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:12.561026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:12.588143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30284d5b-59b5-412f-84ce-ea85c8858d88 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:12.589553 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd214a2-5a47-41cf-9ec4-c3637bd9d18c None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:12.597061 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:12.624288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed0241-7510-4438-9bb0-4bccae2675d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:12.624903 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed0241-7510-4438-9bb0-4bccae2675d3 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:12.625182 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed0241-7510-4438-9bb0-4bccae2675d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:12.625454 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed0241-7510-4438-9bb0-4bccae2675d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:12.626571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed0241-7510-4438-9bb0-4bccae2675d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:12.627980 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed0241-7510-4438-9bb0-4bccae2675d3 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:12.628084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a84ae-7510-42bb-9c1d-e4df461c082a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:12.628647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a84ae-7510-42bb-9c1d-e4df461c082a None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:12.628921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a84ae-7510-42bb-9c1d-e4df461c082a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:12.629186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a84ae-7510-42bb-9c1d-e4df461c082a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:12.629808 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 497/1027] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:57:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:12.635060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed0241-7510-4438-9bb0-4bccae2675d3 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:12.705282 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a84ae-7510-42bb-9c1d-e4df461c082a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:12.706721 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a84ae-7510-42bb-9c1d-e4df461c082a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:12.739250 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed0241-7510-4438-9bb0-4bccae2675d3 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:12.774161 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582b5a6-6cda-4fe0-b350-36ff11934b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:12.774784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582b5a6-6cda-4fe0-b350-36ff11934b6c None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:12.775038 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582b5a6-6cda-4fe0-b350-36ff11934b6c None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:12.775338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582b5a6-6cda-4fe0-b350-36ff11934b6c None admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:12.776097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582b5a6-6cda-4fe0-b350-36ff11934b6c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:12.777637 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582b5a6-6cda-4fe0-b350-36ff11934b6c None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:12.790300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:12.808916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 498/1028] 10.222.0.36 () {60 vars in 1141 bytes} [Fri Sep 2 08:57:12 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 08:57:12.818662 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 531/1029] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:57:12 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 02 08:57:12.824437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582b5a6-6cda-4fe0-b350-36ff11934b6c None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:12.854705 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:12.935672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582b5a6-6cda-4fe0-b350-36ff11934b6c None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:12.972465 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b03399-1ee5-47cb-bbe9-d912a7822df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:12.973100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b03399-1ee5-47cb-bbe9-d912a7822df4 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:12.973653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b03399-1ee5-47cb-bbe9-d912a7822df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:12.973653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b03399-1ee5-47cb-bbe9-d912a7822df4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:12.983909 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b03399-1ee5-47cb-bbe9-d912a7822df4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:12.985352 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b03399-1ee5-47cb-bbe9-d912a7822df4 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:13.000612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-f5b03399-1ee5-47cb-bbe9-d912a7822df4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:57:13.000612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:13.000612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:13.000612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:13.000612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:13.000612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:13.000612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:13.000612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:13.000612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:13.000612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:13.000612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:13.000612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:13.000612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:57:13.000612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:57:13.000612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:57:13.000612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:57:13.001572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:13.001572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:13.001572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:13.001572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:13.001572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:13.001572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:13.001572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:13.001572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:13.001572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:13.001572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:13.001572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:13.001572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:13.001572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:13.001572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:13.001572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:13.001572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:13.001572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:57:13.002518 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:57:13.002518 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:57:13.002518 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:13.002518 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 499/1030] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:57:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:13.008608 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a84ae-7510-42bb-9c1d-e4df461c082a None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:13.112101 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a84ae-7510-42bb-9c1d-e4df461c082a None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:13.145923 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8460bc-9ade-4ad6-a341-706981ddebdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:13.146529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8460bc-9ade-4ad6-a341-706981ddebdb None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:13.146812 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8460bc-9ade-4ad6-a341-706981ddebdb None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:13.147178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8460bc-9ade-4ad6-a341-706981ddebdb None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:13.147824 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8460bc-9ade-4ad6-a341-706981ddebdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:13.148217 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8460bc-9ade-4ad6-a341-706981ddebdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:13.149625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8460bc-9ade-4ad6-a341-706981ddebdb None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:13.162027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 532/1031] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:13.169767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b03399-1ee5-47cb-bbe9-d912a7822df4 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:13.250386 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:13.274035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b03399-1ee5-47cb-bbe9-d912a7822df4 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:13.308806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3db43e-25c5-423b-9bd5-efaee0cf6ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:13.309436 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3db43e-25c5-423b-9bd5-efaee0cf6ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:13.309713 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3db43e-25c5-423b-9bd5-efaee0cf6ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:13.309986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3db43e-25c5-423b-9bd5-efaee0cf6ee9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:13.316901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3db43e-25c5-423b-9bd5-efaee0cf6ee9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:13.318329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3db43e-25c5-423b-9bd5-efaee0cf6ee9 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:13.330033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-5f3db43e-25c5-423b-9bd5-efaee0cf6ee9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:57:13.330033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:13.330033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:13.330033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:13.330033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:13.330033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:13.330033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:13.330033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:13.330033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:13.330033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:13.330033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:13.330033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:13.330033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:57:13.330033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:57:13.330033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:57:13.331006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:57:13.331006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:13.331006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:13.331006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:13.331006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:13.331006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:13.331006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:13.331006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:13.331006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:13.331006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:13.331006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:13.331006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:13.331006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:13.331006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:13.331006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:13.331006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:13.331006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:13.331971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:57:13.331971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:57:13.331971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:57:13.331971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:57:13.331971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:57:13.331971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:57:13.331971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:57:13.331971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:13.335427 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 500/1032] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:57:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:13.337042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8460bc-9ade-4ad6-a341-706981ddebdb None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:13.346536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30ac68-e96d-4aaf-90e9-524d62ae6bdd None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:13.346824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30ac68-e96d-4aaf-90e9-524d62ae6bdd None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:13.347128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e30ac68-e96d-4aaf-90e9-524d62ae6bdd None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:13.348615 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 501/1033] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:57:13.353987 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5265f15-da2e-4c4b-b416-9845a2fbeb26 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:13.354127 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5265f15-da2e-4c4b-b416-9845a2fbeb26 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:13.354349 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5265f15-da2e-4c4b-b416-9845a2fbeb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:13.373570 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-b5265f15-da2e-4c4b-b416-9845a2fbeb26 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:13.374684 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b5265f15-da2e-4c4b-b416-9845a2fbeb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:13.379007 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:13.446841 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8460bc-9ade-4ad6-a341-706981ddebdb None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:13.455440 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 502/1034] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:13 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 08:57:13.480834 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88beeb2a-41bb-44dc-9bbd-22772d71bbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:13.481439 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88beeb2a-41bb-44dc-9bbd-22772d71bbbd None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:13.481717 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88beeb2a-41bb-44dc-9bbd-22772d71bbbd None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:13.481988 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88beeb2a-41bb-44dc-9bbd-22772d71bbbd None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:13.482665 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88beeb2a-41bb-44dc-9bbd-22772d71bbbd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:13.482967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88beeb2a-41bb-44dc-9bbd-22772d71bbbd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:13.484424 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88beeb2a-41bb-44dc-9bbd-22772d71bbbd None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:13.501613 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 533/1035] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:57:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:13.503026 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5265f15-da2e-4c4b-b416-9845a2fbeb26 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:13.568100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:13.601933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5265f15-da2e-4c4b-b416-9845a2fbeb26 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:13.636080 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8e17a-0356-4d3f-abe4-a075b40179c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:13.636611 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8e17a-0356-4d3f-abe4-a075b40179c9 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:13.636887 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8e17a-0356-4d3f-abe4-a075b40179c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:13.637156 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8e17a-0356-4d3f-abe4-a075b40179c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:13.637935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8e17a-0356-4d3f-abe4-a075b40179c9 None admin] RBAC: Adding query filter params (role.id=a604599bb8af4abfaada3a92ac76b1bd, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=cce502a83f5343558c2136c0193e7d85) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:13.638239 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8e17a-0356-4d3f-abe4-a075b40179c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:13.639739 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8e17a-0356-4d3f-abe4-a075b40179c9 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:13.645741 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 503/1036] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:57:13 2022] GET /identity/v3/role_assignments?user.id=cce502a83f5343558c2136c0193e7d85&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=a604599bb8af4abfaada3a92ac76b1bd => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:13.720434 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88beeb2a-41bb-44dc-9bbd-22772d71bbbd None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:13.823892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:13.831920 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88beeb2a-41bb-44dc-9bbd-22772d71bbbd None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:13.866063 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd84f2-d552-4e24-908b-86288d4c9923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:13.866690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd84f2-d552-4e24-908b-86288d4c9923 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:13.866964 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd84f2-d552-4e24-908b-86288d4c9923 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:13.867269 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd84f2-d552-4e24-908b-86288d4c9923 None admin] PATH_INFO: `/v3/services/image` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:13.867870 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd84f2-d552-4e24-908b-86288d4c9923 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:13.869223 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd84f2-d552-4e24-908b-86288d4c9923 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:13.878518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-a6bd84f2-d552-4e24-908b-86288d4c9923 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 08:57:13.878518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:13.878518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:57:13.878518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:57:13.878518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:57:13.878518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:57:13.878518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:57:13.878518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:57:13.878518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 08:57:13.878518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:13.878518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:57:13.878518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:13.878518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:13.878518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:13.878518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:13.878518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:13.878518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:13.879771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:13.879771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:13.879771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:13.879771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:13.879771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:13.879771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:13.879771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:57:13.879771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:57:13.879771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:57:13.879771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:57:13.879771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:13.879771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:13.879771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:13.879771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:13.879771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:13.879771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:13.880627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:13.880627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:13.880627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:13.880627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:13.880627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:13.880627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:13.880627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:13.880627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:13.880627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:13.880627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:13.880627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:57:13.880627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:57:13.880627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 08:57:13.880627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:13.880627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 534/1037] 10.222.0.36 () {56 vars in 1105 bytes} [Fri Sep 2 08:57:13 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:13.885589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8e17a-0356-4d3f-abe4-a075b40179c9 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:13.987062 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8e17a-0356-4d3f-abe4-a075b40179c9 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:14.015611 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:14.021964 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19591b40-a7bb-483b-adf8-4925db99eb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:14.022566 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19591b40-a7bb-483b-adf8-4925db99eb5e None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:14.022837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19591b40-a7bb-483b-adf8-4925db99eb5e None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:14.023124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19591b40-a7bb-483b-adf8-4925db99eb5e None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:14.023745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19591b40-a7bb-483b-adf8-4925db99eb5e None admin] RBAC: Adding query filter params (name=image) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:14.024041 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19591b40-a7bb-483b-adf8-4925db99eb5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:14.025384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19591b40-a7bb-483b-adf8-4925db99eb5e None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:14.034733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 504/1038] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:57:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:14.036250 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd84f2-d552-4e24-908b-86288d4c9923 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:14.140035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd84f2-d552-4e24-908b-86288d4c9923 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:14.172484 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19cebf-8ef1-4cdb-972d-f373787d3e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:14.173602 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19cebf-8ef1-4cdb-972d-f373787d3e64 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:14.173874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19cebf-8ef1-4cdb-972d-f373787d3e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:14.174137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19cebf-8ef1-4cdb-972d-f373787d3e64 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:14.174750 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19cebf-8ef1-4cdb-972d-f373787d3e64 None admin] RBAC: Adding query filter params (type=image) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:14.175042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19cebf-8ef1-4cdb-972d-f373787d3e64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:14.176433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19cebf-8ef1-4cdb-972d-f373787d3e64 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:14.184616 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 535/1039] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:57:14 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:14.187649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19591b40-a7bb-483b-adf8-4925db99eb5e None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:14.287605 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19591b40-a7bb-483b-adf8-4925db99eb5e None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:14.320608 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a901634-baef-40df-b826-9539f9259e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:14.321208 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a901634-baef-40df-b826-9539f9259e61 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:14.321481 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a901634-baef-40df-b826-9539f9259e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:14.321747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a901634-baef-40df-b826-9539f9259e61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:14.322442 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a901634-baef-40df-b826-9539f9259e61 None admin] RBAC: Adding query filter params (interface=public, service_id=a73a199ecdc54796a371f6baee90c115, region_id=RegionOne) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:14.322745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a901634-baef-40df-b826-9539f9259e61 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:14.324116 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a901634-baef-40df-b826-9539f9259e61 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:14.329209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 505/1040] 10.222.0.36 () {56 vars in 1251 bytes} [Fri Sep 2 08:57:14 2022] GET /identity/v3/endpoints?service_id=a73a199ecdc54796a371f6baee90c115&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:14.334515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19cebf-8ef1-4cdb-972d-f373787d3e64 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:14.365632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:14.441829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19cebf-8ef1-4cdb-972d-f373787d3e64 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:14.475580 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a9ea3-8442-4dba-a5e3-da9fb524fca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:14.476417 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a9ea3-8442-4dba-a5e3-da9fb524fca9 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:14.476862 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a9ea3-8442-4dba-a5e3-da9fb524fca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:14.477287 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a9ea3-8442-4dba-a5e3-da9fb524fca9 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:14.478031 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a9ea3-8442-4dba-a5e3-da9fb524fca9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:14.479554 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8a9ea3-8442-4dba-a5e3-da9fb524fca9 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:14.487059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 536/1041] 10.222.0.36 () {56 vars in 1087 bytes} [Fri Sep 2 08:57:14 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 08:57:14.767939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d67ef5-21ce-43ff-8c70-ce72a4db4ad8 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:14.768343 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d67ef5-21ce-43ff-8c70-ce72a4db4ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:14.768524 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d67ef5-21ce-43ff-8c70-ce72a4db4ad8 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:14.769966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 506/1042] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:14.774918 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70cf7f-9ed5-401f-a811-e5d428cd4f98 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:14.775246 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70cf7f-9ed5-401f-a811-e5d428cd4f98 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:14.775520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70cf7f-9ed5-401f-a811-e5d428cd4f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:14.793382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-1b70cf7f-9ed5-401f-a811-e5d428cd4f98 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:14.794553 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-1b70cf7f-9ed5-401f-a811-e5d428cd4f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:14.861488 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 537/1043] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:14 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:15.129528 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d67ef5-21ce-43ff-8c70-ce72a4db4ad8 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:15.191910 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:15.239489 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d67ef5-21ce-43ff-8c70-ce72a4db4ad8 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:15.272084 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e6da36-59be-4b4c-b8a2-a86e09564b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:15.272709 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6da36-59be-4b4c-b8a2-a86e09564b02 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:15.272987 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6da36-59be-4b4c-b8a2-a86e09564b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:15.273255 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e6da36-59be-4b4c-b8a2-a86e09564b02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:15.274364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6da36-59be-4b4c-b8a2-a86e09564b02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:15.275808 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e6da36-59be-4b4c-b8a2-a86e09564b02 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:15.279430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 507/1044] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:57:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:15.283705 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70cf7f-9ed5-401f-a811-e5d428cd4f98 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:15.302071 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:15.395262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70cf7f-9ed5-401f-a811-e5d428cd4f98 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:15.429244 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4871d541-d985-4ef8-9aba-569e9326f621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:15.429908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871d541-d985-4ef8-9aba-569e9326f621 None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:15.430190 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871d541-d985-4ef8-9aba-569e9326f621 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:15.430461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4871d541-d985-4ef8-9aba-569e9326f621 None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:15.431135 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871d541-d985-4ef8-9aba-569e9326f621 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:15.432574 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4871d541-d985-4ef8-9aba-569e9326f621 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:15.463537 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 538/1045] 10.222.0.36 () {60 vars in 1132 bytes} [Fri Sep 2 08:57:15 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:57:15.523968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200fc80-2d4b-4faf-b8b9-3dfe48f02288 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:15.524311 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200fc80-2d4b-4faf-b8b9-3dfe48f02288 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:15.524522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200fc80-2d4b-4faf-b8b9-3dfe48f02288 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:15.526071 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 508/1046] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:15.531459 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd11c2-9113-416d-b001-01cb7512c664 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:15.531747 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd11c2-9113-416d-b001-01cb7512c664 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:15.532020 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd11c2-9113-416d-b001-01cb7512c664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:15.551584 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-18cd11c2-9113-416d-b001-01cb7512c664 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:15.552805 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-18cd11c2-9113-416d-b001-01cb7512c664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:15.628888 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 539/1047] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:15.897935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200fc80-2d4b-4faf-b8b9-3dfe48f02288 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:15.999368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200fc80-2d4b-4faf-b8b9-3dfe48f02288 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:16.026484 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:16.035056 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc6bea-864a-4bf8-a501-56362272445b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:16.035714 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc6bea-864a-4bf8-a501-56362272445b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:16.035994 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc6bea-864a-4bf8-a501-56362272445b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:16.036264 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc6bea-864a-4bf8-a501-56362272445b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:16.046308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc6bea-864a-4bf8-a501-56362272445b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:57:16.051661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-56cc6bea-864a-4bf8-a501-56362272445b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 08:57:16.051661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:16.051661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:16.051661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:16.051661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:16.051661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:16.051661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:16.051661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:16.051661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:16.051661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:16.051661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:16.051661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:16.051661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:57:16.051661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:57:16.051661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:57:16.051661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:57:16.052575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:57:16.052575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:57:16.052575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:57:16.052575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:57:16.052575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:16.052575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:16.052575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:16.052575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:16.052575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:16.052575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:16.052575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:16.052575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:16.052575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:16.052575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:16.052575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:16.052575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:16.053445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:16.053445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:16.053445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:16.053445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:16.053445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:57:16.053445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:57:16.053445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:57:16.053445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:57:16.053445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:57:16.053445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:57:16.053445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 08:57:16.053445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:16.055422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 509/1048] 10.222.0.36 () {56 vars in 1120 bytes} [Fri Sep 2 08:57:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:16.058878 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd11c2-9113-416d-b001-01cb7512c664 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:16.061099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:16.159548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd11c2-9113-416d-b001-01cb7512c664 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:16.194233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e778143-5fd5-4b4f-9a1d-11ab25142b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:16.194868 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e778143-5fd5-4b4f-9a1d-11ab25142b78 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:16.195185 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e778143-5fd5-4b4f-9a1d-11ab25142b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:16.195469 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e778143-5fd5-4b4f-9a1d-11ab25142b78 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:16.196127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e778143-5fd5-4b4f-9a1d-11ab25142b78 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:16.196430 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e778143-5fd5-4b4f-9a1d-11ab25142b78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:16.197795 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e778143-5fd5-4b4f-9a1d-11ab25142b78 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:16.205897 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 540/1049] 10.222.0.36 () {56 vars in 1115 bytes} [Fri Sep 2 08:57:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:16.211563 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc6bea-864a-4bf8-a501-56362272445b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:16.312093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc6bea-864a-4bf8-a501-56362272445b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:16.351391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a37050-9852-45b4-9de5-ee3ab4373241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:16.351391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a37050-9852-45b4-9de5-ee3ab4373241 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:16.351797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a37050-9852-45b4-9de5-ee3ab4373241 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:16.351959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a37050-9852-45b4-9de5-ee3ab4373241 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:16.367037 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a37050-9852-45b4-9de5-ee3ab4373241 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:16.368514 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a37050-9852-45b4-9de5-ee3ab4373241 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:16.384417 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-d0a37050-9852-45b4-9de5-ee3ab4373241 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:57:16.384417 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:16.384417 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:16.384417 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:16.384417 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:16.384417 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:16.384417 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:16.384417 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:16.384417 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:16.384417 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:16.384417 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:16.384417 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:16.384417 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:57:16.384417 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:57:16.384417 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:57:16.384417 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:57:16.385391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:16.385391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:16.385391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:16.385391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:16.385391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:16.385391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:16.385391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:16.385391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:16.385391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:16.385391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:16.385391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:16.385391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:16.385391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:16.385391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:16.385391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:16.385391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:16.385391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:57:16.386314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:57:16.386314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:57:16.386314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:16.390265 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 510/1050] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:57:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:16.391875 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e778143-5fd5-4b4f-9a1d-11ab25142b78 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:16.404297 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c051c18-7d6c-4bc6-92c1-7483dc3685f9 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:16.404567 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c051c18-7d6c-4bc6-92c1-7483dc3685f9 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:16.404846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c051c18-7d6c-4bc6-92c1-7483dc3685f9 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:16.411254 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 511/1051] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:57:16.413767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0b5a3-223e-49ff-b71e-1a4bd4be0592 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:16.413767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0b5a3-223e-49ff-b71e-1a4bd4be0592 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:16.413767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0b5a3-223e-49ff-b71e-1a4bd4be0592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:16.437655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-e7d0b5a3-223e-49ff-b71e-1a4bd4be0592 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:16.439490 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-e7d0b5a3-223e-49ff-b71e-1a4bd4be0592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:16.465515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:16.480192 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:16.510813 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e778143-5fd5-4b4f-9a1d-11ab25142b78 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:16.516894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 512/1052] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 08:57:16.551434 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47357d82-985b-40c6-b138-0ecbb320182e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:16.552043 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47357d82-985b-40c6-b138-0ecbb320182e None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:16.552322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47357d82-985b-40c6-b138-0ecbb320182e None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:16.552595 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47357d82-985b-40c6-b138-0ecbb320182e None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:16.553233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47357d82-985b-40c6-b138-0ecbb320182e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:16.553536 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47357d82-985b-40c6-b138-0ecbb320182e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:16.554902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47357d82-985b-40c6-b138-0ecbb320182e None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:16.567255 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 541/1053] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:16.572795 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0b5a3-223e-49ff-b71e-1a4bd4be0592 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:16.674447 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0b5a3-223e-49ff-b71e-1a4bd4be0592 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:16.707842 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5faf6-e14b-4878-a842-a9f9010a9b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:16.708470 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5faf6-e14b-4878-a842-a9f9010a9b99 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:16.708750 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5faf6-e14b-4878-a842-a9f9010a9b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:16.709024 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5faf6-e14b-4878-a842-a9f9010a9b99 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:16.716460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5faf6-e14b-4878-a842-a9f9010a9b99 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:16.717895 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5faf6-e14b-4878-a842-a9f9010a9b99 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:16.729946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-52d5faf6-e14b-4878-a842-a9f9010a9b99 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 08:57:16.729946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:16.729946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:16.729946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:16.729946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:16.729946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:16.729946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:16.729946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:16.729946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:16.729946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:16.729946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:16.729946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:16.729946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:57:16.729946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:57:16.729946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:57:16.729946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:57:16.730933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:16.730933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:16.730933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:57:16.730933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:57:16.730933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:57:16.730933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:57:16.730933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:16.730933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:16.730933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:16.730933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:16.730933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:16.730933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:16.730933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:16.730933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:16.730933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:16.730933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:16.730933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:16.730933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:16.732257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:16.732257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:16.732257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:57:16.732257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:57:16.732257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:57:16.732257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:57:16.732257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:57:16.732257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:57:16.732257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 08:57:16.732257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:16.735968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 513/1054] 10.222.0.36 () {56 vars in 1093 bytes} [Fri Sep 2 08:57:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:16.737699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47357d82-985b-40c6-b138-0ecbb320182e None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:16.790398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5faf6-e14b-4878-a842-a9f9010a9b99 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:16.821412 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:16.846951 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47357d82-985b-40c6-b138-0ecbb320182e None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:16.874374 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:16.885785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bd7fc-27ce-4565-b5d4-bab8b309b1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:16.886393 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bd7fc-27ce-4565-b5d4-bab8b309b1cf None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:16.886674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bd7fc-27ce-4565-b5d4-bab8b309b1cf None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:16.887143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37bd7fc-27ce-4565-b5d4-bab8b309b1cf None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:16.887694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37bd7fc-27ce-4565-b5d4-bab8b309b1cf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:16.888058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37bd7fc-27ce-4565-b5d4-bab8b309b1cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:16.889422 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37bd7fc-27ce-4565-b5d4-bab8b309b1cf None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:16.896666 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5faf6-e14b-4878-a842-a9f9010a9b99 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:16.909101 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 542/1055] 10.222.0.36 () {56 vars in 1133 bytes} [Fri Sep 2 08:57:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:16.914689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bd7fc-27ce-4565-b5d4-bab8b309b1cf None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:16.936020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9569c-6afc-43ea-a35b-33782cbc04e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:16.936991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9569c-6afc-43ea-a35b-33782cbc04e7 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:16.936991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9569c-6afc-43ea-a35b-33782cbc04e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:16.937280 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9569c-6afc-43ea-a35b-33782cbc04e7 None admin] PATH_INFO: `/v3/services/image` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:16.937941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9569c-6afc-43ea-a35b-33782cbc04e7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:16.939557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9569c-6afc-43ea-a35b-33782cbc04e7 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:16.949323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-66e9569c-6afc-43ea-a35b-33782cbc04e7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 08:57:16.949323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:16.949323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:57:16.949323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:57:16.949323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:57:16.949323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:57:16.949323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:57:16.949323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:57:16.949323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 08:57:16.949323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:16.949323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:57:16.949323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:16.949323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:16.949323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:16.949323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:16.949323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:16.949323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:16.955293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:16.955293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:16.955293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:16.955293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:16.955293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:16.955293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:16.955293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:57:16.955293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:57:16.955293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:57:16.955293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:57:16.955293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:16.955293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:16.955293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:16.955293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:16.955293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:16.955293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:16.956258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:16.956258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:16.956258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:16.956258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:16.956258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:16.956258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:16.956258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:16.956258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:16.956258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:16.956258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:16.956258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:57:16.956258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:57:16.956258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 08:57:16.956258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:16.956258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 514/1056] 10.222.0.36 () {56 vars in 1105 bytes} [Fri Sep 2 08:57:16 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:57:16.957082 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9569c-6afc-43ea-a35b-33782cbc04e7 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:17.017960 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37bd7fc-27ce-4565-b5d4-bab8b309b1cf None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:17.053482 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af5500-ea1a-41c1-8108-f74bac85b778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:17.054114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af5500-ea1a-41c1-8108-f74bac85b778 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:17.054396 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af5500-ea1a-41c1-8108-f74bac85b778 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:17.054669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af5500-ea1a-41c1-8108-f74bac85b778 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:17.055486 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:17.063619 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9569c-6afc-43ea-a35b-33782cbc04e7 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:17.065638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af5500-ea1a-41c1-8108-f74bac85b778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:17.067104 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3af5500-ea1a-41c1-8108-f74bac85b778 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:17.082209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-c3af5500-ea1a-41c1-8108-f74bac85b778 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:57:17.082209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:17.082209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:17.082209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:17.082209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:17.082209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:17.082209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:17.082209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:17.082209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:17.082209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:17.082209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:17.082209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:17.082209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:57:17.082209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:57:17.082209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:57:17.082209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:57:17.083461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:17.083461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:17.083461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:17.083461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:17.083461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:17.083461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:17.083461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:17.083461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:17.083461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:17.083461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:17.083461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:17.083461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:17.083461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:17.083461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:17.083461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:17.083461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:17.083461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:57:17.084354 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:57:17.084354 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:57:17.084354 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:17.088427 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 543/1057] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:57:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:57:17.089382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af5500-ea1a-41c1-8108-f74bac85b778 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:17.099923 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673e330-a4fb-4c8e-851a-159c438aafa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:17.100737 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673e330-a4fb-4c8e-851a-159c438aafa1 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:17.101013 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673e330-a4fb-4c8e-851a-159c438aafa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:17.101299 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673e330-a4fb-4c8e-851a-159c438aafa1 None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:17.102020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673e330-a4fb-4c8e-851a-159c438aafa1 None admin] RBAC: Adding query filter params (name=image) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:17.102320 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673e330-a4fb-4c8e-851a-159c438aafa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:17.103787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673e330-a4fb-4c8e-851a-159c438aafa1 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:17.113924 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 515/1058] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:57:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:17.114709 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673e330-a4fb-4c8e-851a-159c438aafa1 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:17.123688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:17.193502 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af5500-ea1a-41c1-8108-f74bac85b778 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:17.220685 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673e330-a4fb-4c8e-851a-159c438aafa1 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:17.228492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e35c25-1397-4a67-b33a-2c474271cf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:17.229104 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e35c25-1397-4a67-b33a-2c474271cf37 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:17.229385 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e35c25-1397-4a67-b33a-2c474271cf37 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:17.229655 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e35c25-1397-4a67-b33a-2c474271cf37 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:17.230294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e35c25-1397-4a67-b33a-2c474271cf37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:17.230596 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e35c25-1397-4a67-b33a-2c474271cf37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:17.231997 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e35c25-1397-4a67-b33a-2c474271cf37 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:17.245132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 544/1059] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:17.250746 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e35c25-1397-4a67-b33a-2c474271cf37 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:17.265262 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dbc0c-2253-4b28-8fc4-cb6f4deb8987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:17.266249 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dbc0c-2253-4b28-8fc4-cb6f4deb8987 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:17.266842 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dbc0c-2253-4b28-8fc4-cb6f4deb8987 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:17.267151 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dbc0c-2253-4b28-8fc4-cb6f4deb8987 None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:17.267938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5dbc0c-2253-4b28-8fc4-cb6f4deb8987 None admin] RBAC: Adding query filter params (type=image) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:17.268258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5dbc0c-2253-4b28-8fc4-cb6f4deb8987 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:17.269931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5dbc0c-2253-4b28-8fc4-cb6f4deb8987 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:17.280860 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 516/1060] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:57:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:17.282692 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dbc0c-2253-4b28-8fc4-cb6f4deb8987 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:17.312470 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:17.350014 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:17.362378 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e35c25-1397-4a67-b33a-2c474271cf37 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:17.397017 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dbc0c-2253-4b28-8fc4-cb6f4deb8987 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:17.401126 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5cc4a-4c73-4dc0-af61-7d824b770d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:17.401761 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5cc4a-4c73-4dc0-af61-7d824b770d83 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:17.402036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5cc4a-4c73-4dc0-af61-7d824b770d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:17.402303 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5cc4a-4c73-4dc0-af61-7d824b770d83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:17.409993 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5cc4a-4c73-4dc0-af61-7d824b770d83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:17.411456 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5cc4a-4c73-4dc0-af61-7d824b770d83 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:17.423532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-bbe5cc4a-4c73-4dc0-af61-7d824b770d83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:57:17.423532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:17.423532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:17.423532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:17.423532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:17.423532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:17.423532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:17.423532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:17.423532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:17.423532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:17.423532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:17.423532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:17.423532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:57:17.423532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:57:17.423532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:57:17.424457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:57:17.424457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:17.424457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:17.424457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:17.424457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:17.424457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:17.424457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:17.424457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:17.424457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:17.424457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:17.424457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:17.424457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:17.424457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:17.424457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:17.424457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:17.424457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:17.424457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:17.426074 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:57:17.426074 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:57:17.426074 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:57:17.426074 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:57:17.426074 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:57:17.426074 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:57:17.426074 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:57:17.426074 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:17.426074 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 545/1061] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:57:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:57:17.430139 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5cc4a-4c73-4dc0-af61-7d824b770d83 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:17.438666 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13360a36-1146-4709-946b-cf115b88ef41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:17.439312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13360a36-1146-4709-946b-cf115b88ef41 None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:17.439595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13360a36-1146-4709-946b-cf115b88ef41 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:17.439869 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13360a36-1146-4709-946b-cf115b88ef41 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:17.440534 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13360a36-1146-4709-946b-cf115b88ef41 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:17.441896 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13360a36-1146-4709-946b-cf115b88ef41 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:17.460923 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 517/1062] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 08:57:17 2022] POST /identity/v3/endpoints => generated 327 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:57:17.462415 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5ba1e-bd0f-458a-baf2-0a15f51a7801 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:17.462711 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5ba1e-bd0f-458a-baf2-0a15f51a7801 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:17.462983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5ba1e-bd0f-458a-baf2-0a15f51a7801 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:17.464456 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 518/1063] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:17.470996 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce28fc-2aec-454b-84b2-49f6917e1adb None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:17.471307 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce28fc-2aec-454b-84b2-49f6917e1adb None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:17.471581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce28fc-2aec-454b-84b2-49f6917e1adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:17.493848 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-5dce28fc-2aec-454b-84b2-49f6917e1adb None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:17.495264 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-5dce28fc-2aec-454b-84b2-49f6917e1adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:17.544723 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5cc4a-4c73-4dc0-af61-7d824b770d83 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:17.581438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 519/1064] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 08:57:17.590195 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721797c-ab4a-4a94-949b-db090d5d91b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:17.590908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721797c-ab4a-4a94-949b-db090d5d91b8 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:17.591291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721797c-ab4a-4a94-949b-db090d5d91b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:17.591626 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721797c-ab4a-4a94-949b-db090d5d91b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:17.592435 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7721797c-ab4a-4a94-949b-db090d5d91b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:17.592805 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7721797c-ab4a-4a94-949b-db090d5d91b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:17.594304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7721797c-ab4a-4a94-949b-db090d5d91b8 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:17.607727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 546/1065] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:57:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:17.614034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce28fc-2aec-454b-84b2-49f6917e1adb None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:17.617616 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:17.716890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce28fc-2aec-454b-84b2-49f6917e1adb None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:17.750629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8c7aa-88cc-4e85-a885-d10034c2783c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:17.751346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8c7aa-88cc-4e85-a885-d10034c2783c None admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:17.751622 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8c7aa-88cc-4e85-a885-d10034c2783c None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:17.751887 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8c7aa-88cc-4e85-a885-d10034c2783c None admin] PATH_INFO: `/v3/projects/eef80493db09437b88f2b147a59928c1/users/cce502a83f5343558c2136c0193e7d85/roles/a604599bb8af4abfaada3a92ac76b1bd` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:17.754040 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8c7aa-88cc-4e85-a885-d10034c2783c None admin] RBAC: Authorizing `identity:create_grant(project_id=eef80493db09437b88f2b147a59928c1, user_id=cce502a83f5343558c2136c0193e7d85, role_id=a604599bb8af4abfaada3a92ac76b1bd)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:17.755496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8c7aa-88cc-4e85-a885-d10034c2783c None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:17.769001 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 520/1066] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:57:17 2022] PUT /identity/v3/projects/eef80493db09437b88f2b147a59928c1/users/cce502a83f5343558c2136c0193e7d85/roles/a604599bb8af4abfaada3a92ac76b1bd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:57:17.849854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721797c-ab4a-4a94-949b-db090d5d91b8 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:17.867240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:17.962521 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721797c-ab4a-4a94-949b-db090d5d91b8 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:17.996675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea03a3-77ef-4a5e-b29d-b977b69bc2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:17.997218 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea03a3-77ef-4a5e-b29d-b977b69bc2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:17.997492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea03a3-77ef-4a5e-b29d-b977b69bc2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:17.997754 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea03a3-77ef-4a5e-b29d-b977b69bc2b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:18.009106 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea03a3-77ef-4a5e-b29d-b977b69bc2b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:57:18.013807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-6aea03a3-77ef-4a5e-b29d-b977b69bc2b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:57:18.013807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:18.013807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:18.013807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:18.013807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:18.013807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:18.013807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:18.013807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:18.013807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:18.013807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:18.013807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:18.013807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:18.013807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:57:18.013807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:57:18.013807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:57:18.013807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:57:18.014938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:57:18.014938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:57:18.014938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:57:18.014938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:57:18.014938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:18.014938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:18.014938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:18.014938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:18.014938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:18.014938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:18.014938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:18.014938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:18.014938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:18.014938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:18.014938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:18.014938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:18.016026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:18.016026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:18.016026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:18.016026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:18.016026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:57:18.016026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:57:18.016026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:57:18.016026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:57:18.016026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:57:18.016026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:57:18.016026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:57:18.016026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:18.016026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 547/1067] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:57:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:18.022643 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8c7aa-88cc-4e85-a885-d10034c2783c None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:18.085995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:18.124613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8c7aa-88cc-4e85-a885-d10034c2783c None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:18.159574 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9dc70a-774c-46da-95aa-6d78b41e6e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:18.160338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9dc70a-774c-46da-95aa-6d78b41e6e1b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:18.160718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9dc70a-774c-46da-95aa-6d78b41e6e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:18.161093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9dc70a-774c-46da-95aa-6d78b41e6e1b None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:18.161861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9dc70a-774c-46da-95aa-6d78b41e6e1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:18.162265 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9dc70a-774c-46da-95aa-6d78b41e6e1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:18.163772 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9dc70a-774c-46da-95aa-6d78b41e6e1b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:18.172427 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 521/1068] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:57:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:18.178530 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea03a3-77ef-4a5e-b29d-b977b69bc2b5 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:18.255109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:18.280518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea03a3-77ef-4a5e-b29d-b977b69bc2b5 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:18.315641 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743a5e9-8560-478c-bd35-20c586c8a0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:18.316266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743a5e9-8560-478c-bd35-20c586c8a0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:18.316546 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743a5e9-8560-478c-bd35-20c586c8a0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:18.316820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743a5e9-8560-478c-bd35-20c586c8a0f4 None admin] PATH_INFO: `/v3/users/fb1bffc3fc2b499ca5f9d770b2b9d1e7` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:18.330444 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743a5e9-8560-478c-bd35-20c586c8a0f4 None admin] RBAC: Authorizing `identity:get_user(user_id=fb1bffc3fc2b499ca5f9d770b2b9d1e7)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:18.331918 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743a5e9-8560-478c-bd35-20c586c8a0f4 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:18.333715 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 548/1069] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:57:18 2022] GET /identity/v3/users/fb1bffc3fc2b499ca5f9d770b2b9d1e7 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:18.343102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9dc70a-774c-46da-95aa-6d78b41e6e1b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:18.443529 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9dc70a-774c-46da-95aa-6d78b41e6e1b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:18.468983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:18.479171 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a65e0b-91d7-4be0-9c6b-3abba4051cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:18.479854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a65e0b-91d7-4be0-9c6b-3abba4051cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:18.480323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a65e0b-91d7-4be0-9c6b-3abba4051cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:18.480470 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a65e0b-91d7-4be0-9c6b-3abba4051cb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:18.547624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a65e0b-91d7-4be0-9c6b-3abba4051cb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:18.549024 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a65e0b-91d7-4be0-9c6b-3abba4051cb0 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:18.651243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 522/1070] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:57:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 08:57:18.655801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743a5e9-8560-478c-bd35-20c586c8a0f4 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:18.756644 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743a5e9-8560-478c-bd35-20c586c8a0f4 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:18.786552 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:18.792792 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f055bc0-22a9-441d-9727-f07c840e015d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:18.793736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f055bc0-22a9-441d-9727-f07c840e015d None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:18.794152 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f055bc0-22a9-441d-9727-f07c840e015d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:18.794522 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f055bc0-22a9-441d-9727-f07c840e015d None admin] PATH_INFO: `/v3/projects/6a1855dabbc14c19acdb46018883fb4f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:18.795899 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f055bc0-22a9-441d-9727-f07c840e015d None admin] RBAC: Authorizing `identity:get_project(project_id=6a1855dabbc14c19acdb46018883fb4f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:18.797465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f055bc0-22a9-441d-9727-f07c840e015d None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:18.804373 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 549/1071] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:57:18 2022] GET /identity/v3/projects/6a1855dabbc14c19acdb46018883fb4f => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:18.805583 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a65e0b-91d7-4be0-9c6b-3abba4051cb0 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:18.834201 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:18.911002 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a65e0b-91d7-4be0-9c6b-3abba4051cb0 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:18.945033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15575801-824b-4d82-a785-413805b39ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:18.945637 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15575801-824b-4d82-a785-413805b39ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:18.945917 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15575801-824b-4d82-a785-413805b39ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:18.946188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15575801-824b-4d82-a785-413805b39ff8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:18.946905 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15575801-824b-4d82-a785-413805b39ff8 None admin] RBAC: Adding query filter params (role.id=a0a76de72fba4036bc1e3e64df717c7f, scope.project.id=6a1855dabbc14c19acdb46018883fb4f, user.id=fb1bffc3fc2b499ca5f9d770b2b9d1e7) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:18.947245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15575801-824b-4d82-a785-413805b39ff8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:18.948718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15575801-824b-4d82-a785-413805b39ff8 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:18.954776 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 523/1072] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:57:18 2022] GET /identity/v3/role_assignments?user.id=fb1bffc3fc2b499ca5f9d770b2b9d1e7&scope.project.id=6a1855dabbc14c19acdb46018883fb4f&role.id=a0a76de72fba4036bc1e3e64df717c7f => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:19.432456 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2d062-7d39-4c69-a0f7-4a583ee8ac93 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:19.432779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2d062-7d39-4c69-a0f7-4a583ee8ac93 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:19.433005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2d062-7d39-4c69-a0f7-4a583ee8ac93 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:19.434649 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 550/1073] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:19.440296 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e56c9-b473-4a3d-9d26-ecebcaebda9a None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:19.440737 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e56c9-b473-4a3d-9d26-ecebcaebda9a None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:19.440899 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e56c9-b473-4a3d-9d26-ecebcaebda9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:19.458605 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-156e56c9-b473-4a3d-9d26-ecebcaebda9a None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:19.459811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-156e56c9-b473-4a3d-9d26-ecebcaebda9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:19.525512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 524/1074] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:19.669619 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58413534-d59b-4c83-9948-a3285e163ad3 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:19.670116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58413534-d59b-4c83-9948-a3285e163ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:19.670116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58413534-d59b-4c83-9948-a3285e163ad3 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:19.675853 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 551/1075] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:19.677428 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be60dfe-c28e-4692-82bd-010b78a56395 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:19.677713 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be60dfe-c28e-4692-82bd-010b78a56395 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:19.677983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be60dfe-c28e-4692-82bd-010b78a56395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:19.696290 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-7be60dfe-c28e-4692-82bd-010b78a56395 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:19.697861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7be60dfe-c28e-4692-82bd-010b78a56395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:19.717621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:19.771200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 525/1076] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:19.789937 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58413534-d59b-4c83-9948-a3285e163ad3 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:19.896257 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58413534-d59b-4c83-9948-a3285e163ad3 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:19.931420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f22a6d-ffbe-4e50-a4f9-59814e07493d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:19.931950 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f22a6d-ffbe-4e50-a4f9-59814e07493d None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:19.932229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f22a6d-ffbe-4e50-a4f9-59814e07493d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:19.932502 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f22a6d-ffbe-4e50-a4f9-59814e07493d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:19.999836 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:20.002580 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f22a6d-ffbe-4e50-a4f9-59814e07493d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:20.004135 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f22a6d-ffbe-4e50-a4f9-59814e07493d None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:20.038198 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be60dfe-c28e-4692-82bd-010b78a56395 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:20.119179 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 552/1077] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:57:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 08:57:20.125087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f22a6d-ffbe-4e50-a4f9-59814e07493d None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:20.159133 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be60dfe-c28e-4692-82bd-010b78a56395 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:20.197875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59690fba-9d3f-4593-ab31-87de16c5db0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:20.198440 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59690fba-9d3f-4593-ab31-87de16c5db0d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:20.198720 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59690fba-9d3f-4593-ab31-87de16c5db0d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:20.198994 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59690fba-9d3f-4593-ab31-87de16c5db0d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:20.210056 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59690fba-9d3f-4593-ab31-87de16c5db0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:57:20.215353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-59690fba-9d3f-4593-ab31-87de16c5db0d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 08:57:20.215353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:20.215353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:20.215353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:20.215353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:20.215353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:20.215353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:20.215353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:20.215353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:20.215353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:20.215353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:20.215353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:20.215353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:57:20.215353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:57:20.215353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:57:20.215353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:57:20.216475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:57:20.216475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:57:20.216475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:57:20.216475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:57:20.216475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:20.216475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:20.216475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:20.216475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:20.216475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:20.216475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:20.216475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:20.216475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:20.216475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:20.216475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:20.216475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:20.216475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:20.217353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:20.217353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:20.217353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:20.217353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:20.217353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:57:20.217353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:57:20.217353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:57:20.217353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:57:20.217353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:57:20.217353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:57:20.217353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 08:57:20.217353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:20.217353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 526/1078] 10.222.0.36 () {56 vars in 1120 bytes} [Fri Sep 2 08:57:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:57:20.222328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59690fba-9d3f-4593-ab31-87de16c5db0d None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:20.234090 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:20.235909 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:20.237427 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f22a6d-ffbe-4e50-a4f9-59814e07493d None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:20.274307 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13573ba4-61f7-4f0a-8876-bb2f601d278c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:20.274935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13573ba4-61f7-4f0a-8876-bb2f601d278c None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:20.275248 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13573ba4-61f7-4f0a-8876-bb2f601d278c None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:20.275522 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13573ba4-61f7-4f0a-8876-bb2f601d278c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:20.285629 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13573ba4-61f7-4f0a-8876-bb2f601d278c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:20.287082 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13573ba4-61f7-4f0a-8876-bb2f601d278c None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:20.303341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-13573ba4-61f7-4f0a-8876-bb2f601d278c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:57:20.303341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:20.303341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:20.303341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:20.303341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:20.303341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:20.303341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:20.303341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:20.303341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:20.303341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:20.303341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:20.303341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:20.303341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:57:20.303341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:57:20.303341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:57:20.303341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:57:20.304352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:20.304352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:20.304352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:20.304352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:20.304352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:20.304352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:20.304352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:20.304352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:20.304352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:20.304352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:20.304352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:20.304352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:20.304352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:20.304352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:20.304352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:20.304352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:20.304352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:57:20.305237 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:57:20.305237 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:57:20.305237 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:20.305424 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 553/1079] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:57:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:20.312345 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13573ba4-61f7-4f0a-8876-bb2f601d278c None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:20.330689 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59690fba-9d3f-4593-ab31-87de16c5db0d None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:20.374202 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e4dca-e734-4e4d-bd57-9d736e68660e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:20.374963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e4dca-e734-4e4d-bd57-9d736e68660e None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:20.375267 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e4dca-e734-4e4d-bd57-9d736e68660e None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:20.376380 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e4dca-e734-4e4d-bd57-9d736e68660e None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:20.377061 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e4dca-e734-4e4d-bd57-9d736e68660e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:20.377354 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e4dca-e734-4e4d-bd57-9d736e68660e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:20.378703 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e4dca-e734-4e4d-bd57-9d736e68660e None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:20.386733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 527/1080] 10.222.0.36 () {56 vars in 1115 bytes} [Fri Sep 2 08:57:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:20.392590 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e4dca-e734-4e4d-bd57-9d736e68660e None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:20.423775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13573ba4-61f7-4f0a-8876-bb2f601d278c None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:20.463134 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ed1c5-060a-47ae-b465-10ca491211d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:20.463671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ed1c5-060a-47ae-b465-10ca491211d6 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:20.464262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ed1c5-060a-47ae-b465-10ca491211d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:20.464262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ed1c5-060a-47ae-b465-10ca491211d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:20.464858 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ed1c5-060a-47ae-b465-10ca491211d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:20.465160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ed1c5-060a-47ae-b465-10ca491211d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:20.466535 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ed1c5-060a-47ae-b465-10ca491211d6 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:20.474197 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:20.487103 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 554/1081] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:20.517685 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e4dca-e734-4e4d-bd57-9d736e68660e None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:20.554391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ebaaad-5910-494d-aadd-74fdd271c064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:20.555028 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebaaad-5910-494d-aadd-74fdd271c064 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:20.555331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebaaad-5910-494d-aadd-74fdd271c064 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:20.555597 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ebaaad-5910-494d-aadd-74fdd271c064 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:20.568043 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ebaaad-5910-494d-aadd-74fdd271c064 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:20.569492 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ebaaad-5910-494d-aadd-74fdd271c064 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:20.584690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-e2ebaaad-5910-494d-aadd-74fdd271c064 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:57:20.584690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:20.584690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:20.584690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:20.584690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:20.584690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:20.584690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:20.584690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:20.584690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:20.584690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:20.584690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:20.584690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:20.584690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:57:20.584690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:57:20.584690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:57:20.584690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:57:20.585742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:20.585742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:20.585742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:20.585742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:20.585742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:20.585742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:20.585742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:20.585742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:20.585742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:20.585742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:20.585742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:20.585742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:20.585742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:20.585742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:20.585742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:20.585742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:20.585742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:57:20.586650 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:57:20.586650 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:57:20.586650 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:20.590969 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 528/1082] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:57:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:57:20.592361 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ed1c5-060a-47ae-b465-10ca491211d6 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:20.608241 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:20.698960 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ed1c5-060a-47ae-b465-10ca491211d6 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:20.736735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c27adb6-b2cb-4750-801b-67f6bb53e691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:20.737266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27adb6-b2cb-4750-801b-67f6bb53e691 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:20.737546 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27adb6-b2cb-4750-801b-67f6bb53e691 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:20.737818 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c27adb6-b2cb-4750-801b-67f6bb53e691 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:20.738455 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c27adb6-b2cb-4750-801b-67f6bb53e691 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:20.738756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c27adb6-b2cb-4750-801b-67f6bb53e691 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:20.740163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c27adb6-b2cb-4750-801b-67f6bb53e691 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:20.753001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 555/1083] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:57:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:20.758780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ebaaad-5910-494d-aadd-74fdd271c064 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:20.860876 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45f652-6057-44e6-9945-ce0275524f4b None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:20.861165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45f652-6057-44e6-9945-ce0275524f4b None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:20.861439 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a45f652-6057-44e6-9945-ce0275524f4b None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:20.861690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ebaaad-5910-494d-aadd-74fdd271c064 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:20.862786 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 556/1084] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:20.867638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33183e5e-37d9-4966-af64-8f111daf976d None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:20.867918 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33183e5e-37d9-4966-af64-8f111daf976d None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:20.868193 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33183e5e-37d9-4966-af64-8f111daf976d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:20.885568 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:20.886396 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-33183e5e-37d9-4966-af64-8f111daf976d None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:20.887707 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-33183e5e-37d9-4966-af64-8f111daf976d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:20.896629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae17b5-c5fb-40fb-b2a0-10d413b94ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:20.897254 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae17b5-c5fb-40fb-b2a0-10d413b94ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:20.897534 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae17b5-c5fb-40fb-b2a0-10d413b94ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:20.897808 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae17b5-c5fb-40fb-b2a0-10d413b94ba0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:20.905519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae17b5-c5fb-40fb-b2a0-10d413b94ba0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:20.906949 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcae17b5-c5fb-40fb-b2a0-10d413b94ba0 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:20.918938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-dcae17b5-c5fb-40fb-b2a0-10d413b94ba0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 08:57:20.918938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:20.918938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:20.918938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:20.918938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:20.918938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:20.918938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:20.918938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:20.918938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:20.918938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:20.918938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:20.918938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:20.918938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:57:20.918938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:57:20.918938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:57:20.918938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:57:20.919941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:20.919941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:20.919941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:57:20.919941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:57:20.919941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:57:20.919941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:57:20.919941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:20.919941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:20.919941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:20.919941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:20.919941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:20.919941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:20.919941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:20.919941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:20.919941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:20.919941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:20.919941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:20.919941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:20.920882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:20.920882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:20.920882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:57:20.920882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:57:20.920882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:57:20.920882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:57:20.920882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:57:20.920882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:57:20.920882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 08:57:20.920882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:20.920882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 529/1085] 10.222.0.36 () {56 vars in 1093 bytes} [Fri Sep 2 08:57:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:20.926418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae17b5-c5fb-40fb-b2a0-10d413b94ba0 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:20.955598 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 557/1086] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:21.023176 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae17b5-c5fb-40fb-b2a0-10d413b94ba0 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:21.055271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b951f-6a9f-43cc-ab8c-cc017a206adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:21.055959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b951f-6a9f-43cc-ab8c-cc017a206adf None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:21.056239 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b951f-6a9f-43cc-ab8c-cc017a206adf None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:21.056507 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b951f-6a9f-43cc-ab8c-cc017a206adf None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:21.057222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b951f-6a9f-43cc-ab8c-cc017a206adf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:21.057522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b951f-6a9f-43cc-ab8c-cc017a206adf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:21.059202 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b951f-6a9f-43cc-ab8c-cc017a206adf None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:21.075329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 530/1087] 10.222.0.36 () {56 vars in 1133 bytes} [Fri Sep 2 08:57:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:21.080976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33183e5e-37d9-4966-af64-8f111daf976d None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:21.083616 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:21.182961 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33183e5e-37d9-4966-af64-8f111daf976d None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:21.215925 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d4fd4-72ef-45e9-ae4d-edc122397392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:21.216526 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d4fd4-72ef-45e9-ae4d-edc122397392 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:21.216805 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d4fd4-72ef-45e9-ae4d-edc122397392 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:21.217075 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d4fd4-72ef-45e9-ae4d-edc122397392 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:21.218662 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b951f-6a9f-43cc-ab8c-cc017a206adf None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:21.261624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:21.285167 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d4fd4-72ef-45e9-ae4d-edc122397392 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:21.286558 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d4fd4-72ef-45e9-ae4d-edc122397392 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:21.288380 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:21.318167 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b951f-6a9f-43cc-ab8c-cc017a206adf None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:21.350900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3abf0-2933-4499-8471-b1b399294b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:21.351560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3abf0-2933-4499-8471-b1b399294b9c None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:21.351831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3abf0-2933-4499-8471-b1b399294b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:21.352106 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3abf0-2933-4499-8471-b1b399294b9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:21.363359 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3abf0-2933-4499-8471-b1b399294b9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:57:21.368121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-a6a3abf0-2933-4499-8471-b1b399294b9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:57:21.368121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:21.368121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:21.368121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:21.368121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:21.368121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:21.368121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:21.368121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:21.368121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:21.368121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:21.368121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:21.368121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:21.368121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:57:21.368121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:57:21.368121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:57:21.368121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:57:21.369245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:57:21.369245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:57:21.369245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:57:21.369245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:57:21.369245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:21.369245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:21.369245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:21.369245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:21.369245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:21.369245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:21.369245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:21.369245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:21.369245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:21.369245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:21.369245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:21.369245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:21.370277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:21.370277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:21.370277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:21.370277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:21.370277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:57:21.370277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:57:21.370277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:57:21.370277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:57:21.370277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:57:21.370277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:57:21.370277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:57:21.370277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:21.370277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 531/1088] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:57:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:57:21.375090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3abf0-2933-4499-8471-b1b399294b9c None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:21.386881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 558/1089] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:57:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 08:57:21.392836 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d4fd4-72ef-45e9-ae4d-edc122397392 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:21.473261 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3abf0-2933-4499-8471-b1b399294b9c None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:21.474769 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:21.488137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d4fd4-72ef-45e9-ae4d-edc122397392 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:21.489639 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:21.507631 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74696339-e1f5-4bd4-8e7d-c01fb76676de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:21.508230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74696339-e1f5-4bd4-8e7d-c01fb76676de None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:21.508508 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74696339-e1f5-4bd4-8e7d-c01fb76676de None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:21.508778 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74696339-e1f5-4bd4-8e7d-c01fb76676de None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:21.509430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74696339-e1f5-4bd4-8e7d-c01fb76676de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:21.509731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74696339-e1f5-4bd4-8e7d-c01fb76676de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:21.511105 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74696339-e1f5-4bd4-8e7d-c01fb76676de None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:21.519436 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 532/1090] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:57:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:21.523563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5abdb8-abb1-47c8-bae6-f6a5d8cf58e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:21.524192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5abdb8-abb1-47c8-bae6-f6a5d8cf58e8 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:21.524473 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5abdb8-abb1-47c8-bae6-f6a5d8cf58e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:21.524743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5abdb8-abb1-47c8-bae6-f6a5d8cf58e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:21.525064 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74696339-e1f5-4bd4-8e7d-c01fb76676de None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:21.535744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5abdb8-abb1-47c8-bae6-f6a5d8cf58e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:21.537125 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5abdb8-abb1-47c8-bae6-f6a5d8cf58e8 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:21.551688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-ef5abdb8-abb1-47c8-bae6-f6a5d8cf58e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:57:21.551688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:21.551688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:21.551688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:21.551688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:21.551688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:21.551688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:21.551688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:21.551688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:21.551688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:21.551688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:21.551688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:21.551688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:57:21.551688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:57:21.551688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:57:21.551688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:57:21.552801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:21.552801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:21.552801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:21.552801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:21.552801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:21.552801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:21.552801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:21.552801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:21.552801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:21.552801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:21.552801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:21.552801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:21.552801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:21.552801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:21.552801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:21.552801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:21.552801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:57:21.553697 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:57:21.553697 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:57:21.553697 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:21.553697 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 559/1091] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:57:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:57:21.558211 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5abdb8-abb1-47c8-bae6-f6a5d8cf58e8 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:21.621068 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74696339-e1f5-4bd4-8e7d-c01fb76676de None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:21.653131 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1b178-605e-41fc-abd8-136f97eaf65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:21.653761 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1b178-605e-41fc-abd8-136f97eaf65e None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:21.654043 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1b178-605e-41fc-abd8-136f97eaf65e None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:21.654312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1b178-605e-41fc-abd8-136f97eaf65e None admin] PATH_INFO: `/v3/users/fb1bffc3fc2b499ca5f9d770b2b9d1e7` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:21.655379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5abdb8-abb1-47c8-bae6-f6a5d8cf58e8 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:21.655502 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac1b178-605e-41fc-abd8-136f97eaf65e None admin] RBAC: Authorizing `identity:get_user(user_id=fb1bffc3fc2b499ca5f9d770b2b9d1e7)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:21.656901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac1b178-605e-41fc-abd8-136f97eaf65e None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:21.659060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 533/1092] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:57:21 2022] GET /identity/v3/users/fb1bffc3fc2b499ca5f9d770b2b9d1e7 => generated 285 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:21.664719 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1b178-605e-41fc-abd8-136f97eaf65e None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:21.690210 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbaa59-f69a-4b8d-826c-0a4a406a1b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:21.690807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbaa59-f69a-4b8d-826c-0a4a406a1b51 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:21.691112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbaa59-f69a-4b8d-826c-0a4a406a1b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:21.691399 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbaa59-f69a-4b8d-826c-0a4a406a1b51 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:21.692028 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdbaa59-f69a-4b8d-826c-0a4a406a1b51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:21.692331 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdbaa59-f69a-4b8d-826c-0a4a406a1b51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:21.693694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdbaa59-f69a-4b8d-826c-0a4a406a1b51 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:21.695024 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:21.707403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 560/1093] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:57:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:21.713063 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbaa59-f69a-4b8d-826c-0a4a406a1b51 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:21.728543 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:21.765818 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1b178-605e-41fc-abd8-136f97eaf65e None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:21.799744 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9aa95f-9ae4-416c-aeb4-c2d8d9ddb791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:21.800422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9aa95f-9ae4-416c-aeb4-c2d8d9ddb791 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:21.800706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9aa95f-9ae4-416c-aeb4-c2d8d9ddb791 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:21.800981 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9aa95f-9ae4-416c-aeb4-c2d8d9ddb791 None admin] PATH_INFO: `/v3/projects/6a1855dabbc14c19acdb46018883fb4f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:21.802100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9aa95f-9ae4-416c-aeb4-c2d8d9ddb791 None admin] RBAC: Authorizing `identity:get_project(project_id=6a1855dabbc14c19acdb46018883fb4f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:21.803567 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9aa95f-9ae4-416c-aeb4-c2d8d9ddb791 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:21.805191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 534/1094] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:57:21 2022] GET /identity/v3/projects/6a1855dabbc14c19acdb46018883fb4f => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:21.811407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9aa95f-9ae4-416c-aeb4-c2d8d9ddb791 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:21.814155 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbaa59-f69a-4b8d-826c-0a4a406a1b51 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:21.847521 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf747cd-3365-40a4-9a32-01367dbd84fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:21.848130 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf747cd-3365-40a4-9a32-01367dbd84fb None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:21.848408 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf747cd-3365-40a4-9a32-01367dbd84fb None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:21.848677 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf747cd-3365-40a4-9a32-01367dbd84fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:21.855148 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf747cd-3365-40a4-9a32-01367dbd84fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:21.856569 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf747cd-3365-40a4-9a32-01367dbd84fb None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:21.867375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-3cf747cd-3365-40a4-9a32-01367dbd84fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:57:21.867375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:21.867375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:21.867375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:21.867375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:21.867375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:21.867375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:21.867375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:21.867375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:21.867375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:21.867375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:21.867375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:21.867375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:57:21.867375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:57:21.867375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:57:21.868315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:57:21.868315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:21.868315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:21.868315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:21.868315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:21.868315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:21.868315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:21.868315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:21.868315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:21.868315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:21.868315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:21.868315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:21.868315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:21.868315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:21.868315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:21.868315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:21.868315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:21.869229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:57:21.869229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:57:21.869229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:57:21.869229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:57:21.869229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:57:21.869229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:57:21.869229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:57:21.869229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:21.869229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 561/1095] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:57:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:21.874232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf747cd-3365-40a4-9a32-01367dbd84fb None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:21.909256 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:21.911428 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9aa95f-9ae4-416c-aeb4-c2d8d9ddb791 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:21.944527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b23553-db4d-4626-9d21-b25373d704b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:21.945176 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b23553-db4d-4626-9d21-b25373d704b5 None admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:21.945449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b23553-db4d-4626-9d21-b25373d704b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:21.945719 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b23553-db4d-4626-9d21-b25373d704b5 None admin] PATH_INFO: `/v3/projects/6a1855dabbc14c19acdb46018883fb4f/users/fb1bffc3fc2b499ca5f9d770b2b9d1e7/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:21.947620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b23553-db4d-4626-9d21-b25373d704b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a1855dabbc14c19acdb46018883fb4f, user_id=fb1bffc3fc2b499ca5f9d770b2b9d1e7, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:21.948534 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:21.949034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b23553-db4d-4626-9d21-b25373d704b5 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:21.961921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 535/1096] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:57:21 2022] PUT /identity/v3/projects/6a1855dabbc14c19acdb46018883fb4f/users/fb1bffc3fc2b499ca5f9d770b2b9d1e7/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 08:57:21.980374 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf747cd-3365-40a4-9a32-01367dbd84fb None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:22.018117 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62bc24d-a76d-4ee2-a1bd-6922ae1976d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:22.018738 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bc24d-a76d-4ee2-a1bd-6922ae1976d4 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:22.019021 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bc24d-a76d-4ee2-a1bd-6922ae1976d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:22.019326 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62bc24d-a76d-4ee2-a1bd-6922ae1976d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:22.020033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62bc24d-a76d-4ee2-a1bd-6922ae1976d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:22.020341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62bc24d-a76d-4ee2-a1bd-6922ae1976d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:22.021814 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62bc24d-a76d-4ee2-a1bd-6922ae1976d4 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:22.034987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 562/1097] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:57:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:22.041455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b23553-db4d-4626-9d21-b25373d704b5 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:22.145943 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b23553-db4d-4626-9d21-b25373d704b5 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:22.178173 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32482b5c-c492-42f7-86f4-25eb117db890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:22.178773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32482b5c-c492-42f7-86f4-25eb117db890 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:22.179053 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32482b5c-c492-42f7-86f4-25eb117db890 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:22.179358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32482b5c-c492-42f7-86f4-25eb117db890 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:22.180080 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32482b5c-c492-42f7-86f4-25eb117db890 None admin] RBAC: Adding query filter params (role.id=a604599bb8af4abfaada3a92ac76b1bd, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=cce502a83f5343558c2136c0193e7d85) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:22.180386 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32482b5c-c492-42f7-86f4-25eb117db890 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:22.181834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32482b5c-c492-42f7-86f4-25eb117db890 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:22.187752 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 536/1098] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:57:22 2022] GET /identity/v3/role_assignments?user.id=cce502a83f5343558c2136c0193e7d85&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=a604599bb8af4abfaada3a92ac76b1bd => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:23.832323 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc242e37-81e3-4bac-8d42-0184135a87a5 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:23.833113 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc242e37-81e3-4bac-8d42-0184135a87a5 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:23.833113 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc242e37-81e3-4bac-8d42-0184135a87a5 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:23.834641 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 563/1099] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:23.839885 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdd17c-194d-4a97-a27c-00622e9d631a None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:23.840283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdd17c-194d-4a97-a27c-00622e9d631a None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:23.840435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fdd17c-194d-4a97-a27c-00622e9d631a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:23.858672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-35fdd17c-194d-4a97-a27c-00622e9d631a None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:23.859880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-35fdd17c-194d-4a97-a27c-00622e9d631a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:23.876413 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:23.925275 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 537/1100] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:24.195819 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc242e37-81e3-4bac-8d42-0184135a87a5 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:24.292784 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc242e37-81e3-4bac-8d42-0184135a87a5 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:24.326078 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44353465-86ee-449b-8d7d-dc9c1cfa955e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:24.326685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44353465-86ee-449b-8d7d-dc9c1cfa955e None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:24.326962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44353465-86ee-449b-8d7d-dc9c1cfa955e None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:24.327261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44353465-86ee-449b-8d7d-dc9c1cfa955e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:24.329067 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:24.337858 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44353465-86ee-449b-8d7d-dc9c1cfa955e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:57:24.342625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-44353465-86ee-449b-8d7d-dc9c1cfa955e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:57:24.342625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:24.342625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:24.342625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:24.342625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:24.342625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:24.342625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:24.342625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:24.342625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:24.342625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:24.342625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:24.342625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:24.342625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:57:24.342625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:57:24.342625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:57:24.342625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:57:24.343787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:57:24.343787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:57:24.343787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:57:24.343787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:57:24.343787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:24.343787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:24.343787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:24.343787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:24.343787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:24.343787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:24.343787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:24.343787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:24.343787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:24.343787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:24.343787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:24.343787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:24.344671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:24.344671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:24.344671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:24.344671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:24.344671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:57:24.344671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:57:24.344671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:57:24.344671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:57:24.344671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:57:24.344671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:57:24.344671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:57:24.344671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:24.344671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 564/1101] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:57:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:24.349681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdd17c-194d-4a97-a27c-00622e9d631a None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:24.446096 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fdd17c-194d-4a97-a27c-00622e9d631a None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:24.477972 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e04fe1-1aa6-42d7-a4c4-14b5405d367e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:24.478568 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e04fe1-1aa6-42d7-a4c4-14b5405d367e None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:24.478848 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e04fe1-1aa6-42d7-a4c4-14b5405d367e None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:24.479148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e04fe1-1aa6-42d7-a4c4-14b5405d367e None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:24.479807 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e04fe1-1aa6-42d7-a4c4-14b5405d367e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:24.480109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e04fe1-1aa6-42d7-a4c4-14b5405d367e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:24.481460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e04fe1-1aa6-42d7-a4c4-14b5405d367e None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:24.489766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 538/1102] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:57:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:24.495439 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44353465-86ee-449b-8d7d-dc9c1cfa955e None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:24.592362 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44353465-86ee-449b-8d7d-dc9c1cfa955e None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:24.626684 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcaefd-b901-41a1-90a3-b092ecbaf6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:24.627311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcaefd-b901-41a1-90a3-b092ecbaf6fb None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:24.627589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcaefd-b901-41a1-90a3-b092ecbaf6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:24.627858 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffcaefd-b901-41a1-90a3-b092ecbaf6fb None admin] PATH_INFO: `/v3/users/fb1bffc3fc2b499ca5f9d770b2b9d1e7` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:24.628952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffcaefd-b901-41a1-90a3-b092ecbaf6fb None admin] RBAC: Authorizing `identity:get_user(user_id=fb1bffc3fc2b499ca5f9d770b2b9d1e7)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:24.630358 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffcaefd-b901-41a1-90a3-b092ecbaf6fb None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:24.631912 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 565/1103] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:57:24 2022] GET /identity/v3/users/fb1bffc3fc2b499ca5f9d770b2b9d1e7 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:24.637603 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e04fe1-1aa6-42d7-a4c4-14b5405d367e None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:24.655528 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:24.737113 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e04fe1-1aa6-42d7-a4c4-14b5405d367e None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:24.768672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f1463-1f1e-44b5-a5a2-6b016b377ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:24.769268 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f1463-1f1e-44b5-a5a2-6b016b377ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:24.769562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f1463-1f1e-44b5-a5a2-6b016b377ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:24.769816 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f1463-1f1e-44b5-a5a2-6b016b377ae1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:24.834157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f1463-1f1e-44b5-a5a2-6b016b377ae1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:24.835613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2f1463-1f1e-44b5-a5a2-6b016b377ae1 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:24.851871 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:24.930349 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 539/1104] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:57:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 08:57:24.936550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcaefd-b901-41a1-90a3-b092ecbaf6fb None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:24.995756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:25.033184 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffcaefd-b901-41a1-90a3-b092ecbaf6fb None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:25.066070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ce35b-d6c9-4032-9beb-ecde321ddae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:25.066685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ce35b-d6c9-4032-9beb-ecde321ddae2 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:25.066963 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ce35b-d6c9-4032-9beb-ecde321ddae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:25.067262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ce35b-d6c9-4032-9beb-ecde321ddae2 None admin] PATH_INFO: `/v3/projects/6a1855dabbc14c19acdb46018883fb4f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:25.068353 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ce35b-d6c9-4032-9beb-ecde321ddae2 None admin] RBAC: Authorizing `identity:get_project(project_id=6a1855dabbc14c19acdb46018883fb4f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:25.069765 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ce35b-d6c9-4032-9beb-ecde321ddae2 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:25.071374 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 566/1105] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:57:24 2022] GET /identity/v3/projects/6a1855dabbc14c19acdb46018883fb4f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:25.077231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f1463-1f1e-44b5-a5a2-6b016b377ae1 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:25.173237 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f1463-1f1e-44b5-a5a2-6b016b377ae1 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:25.188948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:25.206695 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d037f-99f8-4aee-916b-b0d3252c848b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:25.207319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d037f-99f8-4aee-916b-b0d3252c848b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:25.207597 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d037f-99f8-4aee-916b-b0d3252c848b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:25.207864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d037f-99f8-4aee-916b-b0d3252c848b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:25.208581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d037f-99f8-4aee-916b-b0d3252c848b None admin] RBAC: Adding query filter params (role.id=a0a76de72fba4036bc1e3e64df717c7f, scope.project.id=6a1855dabbc14c19acdb46018883fb4f, user.id=fb1bffc3fc2b499ca5f9d770b2b9d1e7) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:25.208886 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d037f-99f8-4aee-916b-b0d3252c848b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:25.210297 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d037f-99f8-4aee-916b-b0d3252c848b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:25.215838 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 540/1106] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:57:25 2022] GET /identity/v3/role_assignments?user.id=fb1bffc3fc2b499ca5f9d770b2b9d1e7&scope.project.id=6a1855dabbc14c19acdb46018883fb4f&role.id=a0a76de72fba4036bc1e3e64df717c7f => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:57:27.114339 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f5dd6-ca04-4765-85f6-a691e3ca9ad9 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:27.114904 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f5dd6-ca04-4765-85f6-a691e3ca9ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:27.114904 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f5dd6-ca04-4765-85f6-a691e3ca9ad9 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:27.116353 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 567/1107] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:27.121513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff56d1-629c-447c-9628-26dceb95a278 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:27.121888 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff56d1-629c-447c-9628-26dceb95a278 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:27.122057 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ff56d1-629c-447c-9628-26dceb95a278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:27.139588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-26ff56d1-629c-447c-9628-26dceb95a278 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:27.140747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-26ff56d1-629c-447c-9628-26dceb95a278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:27.204035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 541/1108] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:27.468651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f5dd6-ca04-4765-85f6-a691e3ca9ad9 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:27.567382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f5dd6-ca04-4765-85f6-a691e3ca9ad9 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:27.595879 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:27.601551 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f29f9-5db5-4a9e-9c85-423d330804db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:27.602192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f29f9-5db5-4a9e-9c85-423d330804db None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:27.602468 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f29f9-5db5-4a9e-9c85-423d330804db None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:27.602734 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f29f9-5db5-4a9e-9c85-423d330804db None admin] PATH_INFO: `/v3/domains/default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:27.603841 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f29f9-5db5-4a9e-9c85-423d330804db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:27.605253 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f29f9-5db5-4a9e-9c85-423d330804db None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:27.606775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 568/1109] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:57:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:27.612374 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff56d1-629c-447c-9628-26dceb95a278 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:27.706904 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ff56d1-629c-447c-9628-26dceb95a278 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:27.717925 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:27.740860 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161aa6c-4b70-4653-bb9b-efa61f4d3657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:27.741401 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161aa6c-4b70-4653-bb9b-efa61f4d3657 None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:27.741676 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161aa6c-4b70-4653-bb9b-efa61f4d3657 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:27.741948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161aa6c-4b70-4653-bb9b-efa61f4d3657 None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:27.742601 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161aa6c-4b70-4653-bb9b-efa61f4d3657 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:27.744054 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161aa6c-4b70-4653-bb9b-efa61f4d3657 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:27.765632 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 542/1110] 10.222.0.36 () {60 vars in 1132 bytes} [Fri Sep 2 08:57:27 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:57:29.692099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c18e6-ba2f-430c-9d7f-0cb519b946a6 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:29.692608 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c18e6-ba2f-430c-9d7f-0cb519b946a6 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:29.692608 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9c18e6-ba2f-430c-9d7f-0cb519b946a6 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:29.693888 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 569/1111] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:29.698927 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe5be7-cf69-465b-a1e2-75c585d8707e None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:29.699352 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe5be7-cf69-465b-a1e2-75c585d8707e None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:29.699510 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afe5be7-cf69-465b-a1e2-75c585d8707e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:29.717716 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-7afe5be7-cf69-465b-a1e2-75c585d8707e None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:29.718902 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7afe5be7-cf69-465b-a1e2-75c585d8707e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:29.782437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 543/1112] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:30.048313 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c18e6-ba2f-430c-9d7f-0cb519b946a6 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:30.147443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9c18e6-ba2f-430c-9d7f-0cb519b946a6 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:30.179966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085db03-3c85-434e-a6f3-96da7fd6bd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:30.180577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085db03-3c85-434e-a6f3-96da7fd6bd25 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:30.180855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085db03-3c85-434e-a6f3-96da7fd6bd25 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:30.181128 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0085db03-3c85-434e-a6f3-96da7fd6bd25 None admin] PATH_INFO: `/v3/roles/9c6920b8e6f744a6a80371b4218605fd` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:30.182501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085db03-3c85-434e-a6f3-96da7fd6bd25 None admin] RBAC: Authorizing `identity:get_role(role_id=9c6920b8e6f744a6a80371b4218605fd)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:30.184145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0085db03-3c85-434e-a6f3-96da7fd6bd25 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:30.185430 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 570/1113] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:57:30 2022] GET /identity/v3/roles/9c6920b8e6f744a6a80371b4218605fd => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:30.191466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe5be7-cf69-465b-a1e2-75c585d8707e None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:30.281447 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:30.287501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afe5be7-cf69-465b-a1e2-75c585d8707e None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:30.321354 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e166e-c911-4edf-a99b-177dd5231bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:30.321916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e166e-c911-4edf-a99b-177dd5231bdb None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:30.322192 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e166e-c911-4edf-a99b-177dd5231bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:30.322462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e166e-c911-4edf-a99b-177dd5231bdb None admin] PATH_INFO: `/v3/users/49bce65de00d4fdaba34bf3cdf3e1a86` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:30.334952 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e166e-c911-4edf-a99b-177dd5231bdb None admin] RBAC: Authorizing `identity:get_user(user_id=49bce65de00d4fdaba34bf3cdf3e1a86)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:30.336427 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e166e-c911-4edf-a99b-177dd5231bdb None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:30.337990 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 544/1114] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:57:30 2022] GET /identity/v3/users/49bce65de00d4fdaba34bf3cdf3e1a86 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:30.343662 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085db03-3c85-434e-a6f3-96da7fd6bd25 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:30.409103 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:30.443376 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0085db03-3c85-434e-a6f3-96da7fd6bd25 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:30.475383 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4ef3d-31ef-4f90-8e12-3f0174f38aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:30.475970 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4ef3d-31ef-4f90-8e12-3f0174f38aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:30.476247 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4ef3d-31ef-4f90-8e12-3f0174f38aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:30.476515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4ef3d-31ef-4f90-8e12-3f0174f38aa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:30.540707 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4ef3d-31ef-4f90-8e12-3f0174f38aa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:30.542155 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4ef3d-31ef-4f90-8e12-3f0174f38aa8 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:30.603209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:30.638228 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 571/1115] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:57:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 08:57:30.644044 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e166e-c911-4edf-a99b-177dd5231bdb None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:30.739915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e166e-c911-4edf-a99b-177dd5231bdb None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:30.771698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6c436a-df04-44a5-8b7f-c4e34cf4fae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:30.772307 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c436a-df04-44a5-8b7f-c4e34cf4fae4 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:30.772586 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c436a-df04-44a5-8b7f-c4e34cf4fae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:30.772856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6c436a-df04-44a5-8b7f-c4e34cf4fae4 None admin] PATH_INFO: `/v3/projects/6a1855dabbc14c19acdb46018883fb4f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:30.773951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6c436a-df04-44a5-8b7f-c4e34cf4fae4 None admin] RBAC: Authorizing `identity:get_project(project_id=6a1855dabbc14c19acdb46018883fb4f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:30.775387 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6c436a-df04-44a5-8b7f-c4e34cf4fae4 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:30.776971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 545/1116] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:57:30 2022] GET /identity/v3/projects/6a1855dabbc14c19acdb46018883fb4f => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:30.783479 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4ef3d-31ef-4f90-8e12-3f0174f38aa8 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:30.880062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4ef3d-31ef-4f90-8e12-3f0174f38aa8 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:30.912559 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7b7ac-7b66-48df-921f-80af7b4bd8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:30.913143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7b7ac-7b66-48df-921f-80af7b4bd8f1 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:30.913423 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7b7ac-7b66-48df-921f-80af7b4bd8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:30.913694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7b7ac-7b66-48df-921f-80af7b4bd8f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:30.914402 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7b7ac-7b66-48df-921f-80af7b4bd8f1 None admin] RBAC: Adding query filter params (role.id=9c6920b8e6f744a6a80371b4218605fd, scope.project.id=6a1855dabbc14c19acdb46018883fb4f, user.id=49bce65de00d4fdaba34bf3cdf3e1a86) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:30.914709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7b7ac-7b66-48df-921f-80af7b4bd8f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:30.916154 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf7b7ac-7b66-48df-921f-80af7b4bd8f1 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:30.921313 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 572/1117] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:57:30 2022] GET /identity/v3/role_assignments?user.id=49bce65de00d4fdaba34bf3cdf3e1a86&scope.project.id=6a1855dabbc14c19acdb46018883fb4f&role.id=9c6920b8e6f744a6a80371b4218605fd => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:32.794331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03cd89-c1ef-40fb-a7c4-0cc171e301b6 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:32.794878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03cd89-c1ef-40fb-a7c4-0cc171e301b6 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:32.794878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03cd89-c1ef-40fb-a7c4-0cc171e301b6 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:32.796249 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 546/1118] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:32.801737 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490ef4b-5427-4120-97ed-a77f8a6bf1e0 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:32.802144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490ef4b-5427-4120-97ed-a77f8a6bf1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:32.802278 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490ef4b-5427-4120-97ed-a77f8a6bf1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:32.820676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-d490ef4b-5427-4120-97ed-a77f8a6bf1e0 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:32.821825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-d490ef4b-5427-4120-97ed-a77f8a6bf1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:32.844209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:32.886439 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 573/1119] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:33.151671 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03cd89-c1ef-40fb-a7c4-0cc171e301b6 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:33.176266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:33.250705 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03cd89-c1ef-40fb-a7c4-0cc171e301b6 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:33.283237 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c4256-68cf-4253-8deb-bcb6834731b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:33.283849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c4256-68cf-4253-8deb-bcb6834731b4 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:33.284126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c4256-68cf-4253-8deb-bcb6834731b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:33.284398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c4256-68cf-4253-8deb-bcb6834731b4 None admin] PATH_INFO: `/v3/roles/9c6920b8e6f744a6a80371b4218605fd` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:33.285775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c4256-68cf-4253-8deb-bcb6834731b4 None admin] RBAC: Authorizing `identity:get_role(role_id=9c6920b8e6f744a6a80371b4218605fd)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:33.287153 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156c4256-68cf-4253-8deb-bcb6834731b4 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:33.288448 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 547/1120] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:57:33 2022] GET /identity/v3/roles/9c6920b8e6f744a6a80371b4218605fd => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:33.294105 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490ef4b-5427-4120-97ed-a77f8a6bf1e0 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:33.394838 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490ef4b-5427-4120-97ed-a77f8a6bf1e0 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:33.427696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50245f0b-a81c-48be-b9e6-729a3c9e4bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:33.428250 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50245f0b-a81c-48be-b9e6-729a3c9e4bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:33.428526 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50245f0b-a81c-48be-b9e6-729a3c9e4bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:33.428793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50245f0b-a81c-48be-b9e6-729a3c9e4bc2 None admin] PATH_INFO: `/v3/users/49bce65de00d4fdaba34bf3cdf3e1a86` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:33.429889 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50245f0b-a81c-48be-b9e6-729a3c9e4bc2 None admin] RBAC: Authorizing `identity:get_user(user_id=49bce65de00d4fdaba34bf3cdf3e1a86)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:33.431345 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50245f0b-a81c-48be-b9e6-729a3c9e4bc2 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:33.432895 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 574/1121] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:57:33 2022] GET /identity/v3/users/49bce65de00d4fdaba34bf3cdf3e1a86 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:33.439045 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c4256-68cf-4253-8deb-bcb6834731b4 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:33.525873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:33.536033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c4256-68cf-4253-8deb-bcb6834731b4 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:33.568125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ea3776-5f63-47b6-a549-fad15bd674db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:33.568740 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea3776-5f63-47b6-a549-fad15bd674db None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:33.569020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea3776-5f63-47b6-a549-fad15bd674db None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:33.569292 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ea3776-5f63-47b6-a549-fad15bd674db None admin] PATH_INFO: `/v3/projects/6a1855dabbc14c19acdb46018883fb4f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:33.570391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea3776-5f63-47b6-a549-fad15bd674db None admin] RBAC: Authorizing `identity:get_project(project_id=6a1855dabbc14c19acdb46018883fb4f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:33.571838 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ea3776-5f63-47b6-a549-fad15bd674db None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:33.573444 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 548/1122] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:57:33 2022] GET /identity/v3/projects/6a1855dabbc14c19acdb46018883fb4f => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:33.579687 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50245f0b-a81c-48be-b9e6-729a3c9e4bc2 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:33.607116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:33.675692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50245f0b-a81c-48be-b9e6-729a3c9e4bc2 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:33.708361 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e14da6c-e1b3-4eec-a53f-e958b3799717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:33.708940 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14da6c-e1b3-4eec-a53f-e958b3799717 None admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:33.709218 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14da6c-e1b3-4eec-a53f-e958b3799717 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:33.709489 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14da6c-e1b3-4eec-a53f-e958b3799717 None admin] PATH_INFO: `/v3/projects/6a1855dabbc14c19acdb46018883fb4f/users/49bce65de00d4fdaba34bf3cdf3e1a86/roles/9c6920b8e6f744a6a80371b4218605fd` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:33.711592 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e14da6c-e1b3-4eec-a53f-e958b3799717 None admin] RBAC: Authorizing `identity:create_grant(project_id=6a1855dabbc14c19acdb46018883fb4f, user_id=49bce65de00d4fdaba34bf3cdf3e1a86, role_id=9c6920b8e6f744a6a80371b4218605fd)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:33.713007 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e14da6c-e1b3-4eec-a53f-e958b3799717 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:33.727554 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 575/1123] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:57:33 2022] PUT /identity/v3/projects/6a1855dabbc14c19acdb46018883fb4f/users/49bce65de00d4fdaba34bf3cdf3e1a86/roles/9c6920b8e6f744a6a80371b4218605fd => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:57:35.596382 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb7776-bdec-4c58-a02e-11be9f607735 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:35.596930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb7776-bdec-4c58-a02e-11be9f607735 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:35.596930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb7776-bdec-4c58-a02e-11be9f607735 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:35.598265 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 549/1124] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:35.603497 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d390d9-d72f-4507-b092-128224f2e121 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:35.603887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d390d9-d72f-4507-b092-128224f2e121 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:35.604048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d390d9-d72f-4507-b092-128224f2e121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:35.622248 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-c1d390d9-d72f-4507-b092-128224f2e121 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:35.623389 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c1d390d9-d72f-4507-b092-128224f2e121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:35.688875 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 576/1125] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:35.959248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb7776-bdec-4c58-a02e-11be9f607735 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:36.056686 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb7776-bdec-4c58-a02e-11be9f607735 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:36.088498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24266fa7-fbd8-4fad-9d2a-b315ab561e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:36.089111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24266fa7-fbd8-4fad-9d2a-b315ab561e8d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:36.089386 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24266fa7-fbd8-4fad-9d2a-b315ab561e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:36.089653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24266fa7-fbd8-4fad-9d2a-b315ab561e8d None admin] PATH_INFO: `/v3/roles/9c6920b8e6f744a6a80371b4218605fd` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:36.091012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24266fa7-fbd8-4fad-9d2a-b315ab561e8d None admin] RBAC: Authorizing `identity:get_role(role_id=9c6920b8e6f744a6a80371b4218605fd)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:36.092386 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24266fa7-fbd8-4fad-9d2a-b315ab561e8d None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:36.093671 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 550/1126] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:57:35 2022] GET /identity/v3/roles/9c6920b8e6f744a6a80371b4218605fd => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:36.099913 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d390d9-d72f-4507-b092-128224f2e121 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:36.118200 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:36.199571 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d390d9-d72f-4507-b092-128224f2e121 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:36.236724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81a6cf-7baf-4f32-8301-791c7ddedf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:36.237284 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81a6cf-7baf-4f32-8301-791c7ddedf2a None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:36.237563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81a6cf-7baf-4f32-8301-791c7ddedf2a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:36.237833 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81a6cf-7baf-4f32-8301-791c7ddedf2a None admin] PATH_INFO: `/v3/users/49bce65de00d4fdaba34bf3cdf3e1a86` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:36.238943 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81a6cf-7baf-4f32-8301-791c7ddedf2a None admin] RBAC: Authorizing `identity:get_user(user_id=49bce65de00d4fdaba34bf3cdf3e1a86)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:36.240403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81a6cf-7baf-4f32-8301-791c7ddedf2a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:36.241979 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 577/1127] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:57:36 2022] GET /identity/v3/users/49bce65de00d4fdaba34bf3cdf3e1a86 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:36.247774 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24266fa7-fbd8-4fad-9d2a-b315ab561e8d None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:36.273818 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:36.344630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24266fa7-fbd8-4fad-9d2a-b315ab561e8d None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:36.376995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c84544-b11d-4a4c-9280-a696c978845f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:36.377593 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c84544-b11d-4a4c-9280-a696c978845f None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:36.377869 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c84544-b11d-4a4c-9280-a696c978845f None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:36.378139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c84544-b11d-4a4c-9280-a696c978845f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:36.444748 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c84544-b11d-4a4c-9280-a696c978845f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:36.446184 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c84544-b11d-4a4c-9280-a696c978845f None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:36.471761 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:36.543363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 551/1128] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:57:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 08:57:36.549145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81a6cf-7baf-4f32-8301-791c7ddedf2a None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:36.632067 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:36.647167 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81a6cf-7baf-4f32-8301-791c7ddedf2a None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:36.679856 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b574bd1-7d2d-4b88-a918-4c0610424c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:36.680466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b574bd1-7d2d-4b88-a918-4c0610424c3d None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:36.680744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b574bd1-7d2d-4b88-a918-4c0610424c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:36.681015 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b574bd1-7d2d-4b88-a918-4c0610424c3d None admin] PATH_INFO: `/v3/projects/6a1855dabbc14c19acdb46018883fb4f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:36.682113 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b574bd1-7d2d-4b88-a918-4c0610424c3d None admin] RBAC: Authorizing `identity:get_project(project_id=6a1855dabbc14c19acdb46018883fb4f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:36.683837 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b574bd1-7d2d-4b88-a918-4c0610424c3d None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:36.685425 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 578/1129] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:57:36 2022] GET /identity/v3/projects/6a1855dabbc14c19acdb46018883fb4f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:36.691857 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c84544-b11d-4a4c-9280-a696c978845f None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:36.786166 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c84544-b11d-4a4c-9280-a696c978845f None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:36.810143 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:36.819315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d4050-cca8-4f68-85f5-e167377cc7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:36.819902 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d4050-cca8-4f68-85f5-e167377cc7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:36.820178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d4050-cca8-4f68-85f5-e167377cc7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:36.820444 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d4050-cca8-4f68-85f5-e167377cc7b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:36.821151 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d4050-cca8-4f68-85f5-e167377cc7b6 None admin] RBAC: Adding query filter params (role.id=9c6920b8e6f744a6a80371b4218605fd, scope.project.id=6a1855dabbc14c19acdb46018883fb4f, user.id=49bce65de00d4fdaba34bf3cdf3e1a86) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:36.821451 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d4050-cca8-4f68-85f5-e167377cc7b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:36.822853 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d4050-cca8-4f68-85f5-e167377cc7b6 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:36.828367 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 552/1130] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:57:36 2022] GET /identity/v3/role_assignments?user.id=49bce65de00d4fdaba34bf3cdf3e1a86&scope.project.id=6a1855dabbc14c19acdb46018883fb4f&role.id=9c6920b8e6f744a6a80371b4218605fd => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:38.717952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc342bb2-cd57-42f5-9372-d5dd8c6bfff1 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:38.718478 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc342bb2-cd57-42f5-9372-d5dd8c6bfff1 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:38.718478 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc342bb2-cd57-42f5-9372-d5dd8c6bfff1 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:38.719797 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 579/1131] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:38.725043 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746805b-c211-404c-9a36-1c3c56a97cfb None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:38.725455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746805b-c211-404c-9a36-1c3c56a97cfb None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:38.725581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746805b-c211-404c-9a36-1c3c56a97cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:38.743920 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-8746805b-c211-404c-9a36-1c3c56a97cfb None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:38.745097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-8746805b-c211-404c-9a36-1c3c56a97cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:38.809477 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 553/1132] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:39.076508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc342bb2-cd57-42f5-9372-d5dd8c6bfff1 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:39.174583 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc342bb2-cd57-42f5-9372-d5dd8c6bfff1 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:39.207400 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b89459-789c-493c-a568-50c1c8261702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:39.208009 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b89459-789c-493c-a568-50c1c8261702 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:39.208285 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b89459-789c-493c-a568-50c1c8261702 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:39.208553 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b89459-789c-493c-a568-50c1c8261702 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:39.209631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b89459-789c-493c-a568-50c1c8261702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:39.211046 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b89459-789c-493c-a568-50c1c8261702 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:39.212598 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 580/1133] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:57:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:39.218462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746805b-c211-404c-9a36-1c3c56a97cfb None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:39.314137 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746805b-c211-404c-9a36-1c3c56a97cfb None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:39.333758 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:39.348755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42986421-d5c8-461c-8725-6c803bdf52ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:39.349608 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42986421-d5c8-461c-8725-6c803bdf52ce None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:39.349728 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42986421-d5c8-461c-8725-6c803bdf52ce None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:39.349934 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42986421-d5c8-461c-8725-6c803bdf52ce None admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:39.350626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42986421-d5c8-461c-8725-6c803bdf52ce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:39.352074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42986421-d5c8-461c-8725-6c803bdf52ce None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:39.378912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 554/1134] 10.222.0.36 () {60 vars in 1141 bytes} [Fri Sep 2 08:57:39 2022] POST /identity/v3/projects => generated 311 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:57:41.262388 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28058827-3bff-49d5-b479-52f722a0734c None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:41.262948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28058827-3bff-49d5-b479-52f722a0734c None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:41.262948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28058827-3bff-49d5-b479-52f722a0734c None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:41.264306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 581/1135] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:41.269557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb9afa-1893-495c-b312-0de8dd9ba9cf None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:41.269963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb9afa-1893-495c-b312-0de8dd9ba9cf None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:41.270110 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb9afa-1893-495c-b312-0de8dd9ba9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:41.288237 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-1feb9afa-1893-495c-b312-0de8dd9ba9cf None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:41.289412 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1feb9afa-1893-495c-b312-0de8dd9ba9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:41.362468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 555/1136] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:41.633557 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28058827-3bff-49d5-b479-52f722a0734c None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:41.658052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:41.731262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28058827-3bff-49d5-b479-52f722a0734c None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:41.762909 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4885fe8-b949-400b-a6e4-2b4106bc0500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:41.763538 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4885fe8-b949-400b-a6e4-2b4106bc0500 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:41.763815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4885fe8-b949-400b-a6e4-2b4106bc0500 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:41.764084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4885fe8-b949-400b-a6e4-2b4106bc0500 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:41.765167 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4885fe8-b949-400b-a6e4-2b4106bc0500 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:41.767140 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4885fe8-b949-400b-a6e4-2b4106bc0500 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:41.768657 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 582/1137] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:57:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:41.774220 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb9afa-1893-495c-b312-0de8dd9ba9cf None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:41.872123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb9afa-1893-495c-b312-0de8dd9ba9cf None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:41.873663 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:41.907374 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7186619-0edc-4d38-8056-409a0aa572d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:41.907903 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7186619-0edc-4d38-8056-409a0aa572d1 None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:41.908176 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7186619-0edc-4d38-8056-409a0aa572d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:41.908443 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7186619-0edc-4d38-8056-409a0aa572d1 None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:41.909078 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7186619-0edc-4d38-8056-409a0aa572d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:41.910487 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7186619-0edc-4d38-8056-409a0aa572d1 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:41.930013 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 556/1138] 10.222.0.36 () {60 vars in 1132 bytes} [Fri Sep 2 08:57:41 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:57:43.840315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b12539-d83e-4078-ad0e-670c4295ab2f None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:43.840818 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b12539-d83e-4078-ad0e-670c4295ab2f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:43.840818 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b12539-d83e-4078-ad0e-670c4295ab2f None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:43.842115 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 583/1139] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:43.847738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4432a-1258-47e6-948c-22e7ba9e3626 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:43.848188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4432a-1258-47e6-948c-22e7ba9e3626 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:43.848281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4432a-1258-47e6-948c-22e7ba9e3626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:43.866091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-ecc4432a-1258-47e6-948c-22e7ba9e3626 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:43.867305 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ecc4432a-1258-47e6-948c-22e7ba9e3626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:43.931532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 557/1140] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:44.202838 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b12539-d83e-4078-ad0e-670c4295ab2f None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:44.291250 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:44.301823 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b12539-d83e-4078-ad0e-670c4295ab2f None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:44.334874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158ede6-4ed2-43a1-bacc-5837581521eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:44.335514 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158ede6-4ed2-43a1-bacc-5837581521eb None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:44.335790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158ede6-4ed2-43a1-bacc-5837581521eb None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:44.336056 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9158ede6-4ed2-43a1-bacc-5837581521eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:44.345855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9158ede6-4ed2-43a1-bacc-5837581521eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:57:44.350632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-9158ede6-4ed2-43a1-bacc-5837581521eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:57:44.350632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:44.350632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:44.350632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:44.350632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:44.350632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:44.350632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:44.350632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:44.350632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:44.350632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:44.350632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:44.350632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:44.350632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:57:44.350632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:57:44.350632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:57:44.350632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:57:44.351699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:57:44.351699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:57:44.351699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:57:44.351699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:57:44.351699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:44.351699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:44.351699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:44.351699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:44.351699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:44.351699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:44.351699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:44.351699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:44.351699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:44.351699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:44.351699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:44.351699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:44.352640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:44.352640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:44.352640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:44.352640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:44.352640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:57:44.352640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:57:44.352640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:57:44.352640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:57:44.352640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:57:44.352640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:57:44.352640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:57:44.352640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:44.352640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 584/1141] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:57:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:44.357741 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4432a-1258-47e6-948c-22e7ba9e3626 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:44.440638 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:44.453839 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4432a-1258-47e6-948c-22e7ba9e3626 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:44.485346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb999263-80de-4902-b2cb-8f6e17916dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:44.485935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb999263-80de-4902-b2cb-8f6e17916dfb None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:44.486210 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb999263-80de-4902-b2cb-8f6e17916dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:44.486478 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb999263-80de-4902-b2cb-8f6e17916dfb None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:44.487152 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb999263-80de-4902-b2cb-8f6e17916dfb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:44.487452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb999263-80de-4902-b2cb-8f6e17916dfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:44.488794 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb999263-80de-4902-b2cb-8f6e17916dfb None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:44.496459 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 558/1142] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:57:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:44.502618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158ede6-4ed2-43a1-bacc-5837581521eb None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:44.599012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9158ede6-4ed2-43a1-bacc-5837581521eb None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:44.632193 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9d119-3a5b-4524-83d6-70c37274994b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:44.632802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9d119-3a5b-4524-83d6-70c37274994b None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:44.633079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9d119-3a5b-4524-83d6-70c37274994b None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:44.633350 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc9d119-3a5b-4524-83d6-70c37274994b None admin] PATH_INFO: `/v3/users/faa05c33c1744c67bb596b44f9eb725b` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:44.645867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9d119-3a5b-4524-83d6-70c37274994b None admin] RBAC: Authorizing `identity:get_user(user_id=faa05c33c1744c67bb596b44f9eb725b)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:44.647320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc9d119-3a5b-4524-83d6-70c37274994b None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:44.648865 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 585/1143] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:57:44 2022] GET /identity/v3/users/faa05c33c1744c67bb596b44f9eb725b => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:44.654500 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb999263-80de-4902-b2cb-8f6e17916dfb None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:44.752378 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb999263-80de-4902-b2cb-8f6e17916dfb None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:44.783326 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14829820-e277-4a8e-ab24-5d668c1e66fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:44.783921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14829820-e277-4a8e-ab24-5d668c1e66fb None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:44.784197 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14829820-e277-4a8e-ab24-5d668c1e66fb None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:44.784465 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14829820-e277-4a8e-ab24-5d668c1e66fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:44.802449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:44.848964 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14829820-e277-4a8e-ab24-5d668c1e66fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:44.850383 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14829820-e277-4a8e-ab24-5d668c1e66fb None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:44.943645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 559/1144] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:57:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 08:57:44.949572 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9d119-3a5b-4524-83d6-70c37274994b None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:44.960154 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:45.045556 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc9d119-3a5b-4524-83d6-70c37274994b None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:45.078276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d485e-954a-470a-8c16-1c855b0f979c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:45.078891 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d485e-954a-470a-8c16-1c855b0f979c None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:45.079192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d485e-954a-470a-8c16-1c855b0f979c None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:45.079464 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d485e-954a-470a-8c16-1c855b0f979c None admin] PATH_INFO: `/v3/projects/74ec3a9f55124b149ed71539ff22b8b4` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:45.080555 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d485e-954a-470a-8c16-1c855b0f979c None admin] RBAC: Authorizing `identity:get_project(project_id=74ec3a9f55124b149ed71539ff22b8b4)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:45.081980 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d485e-954a-470a-8c16-1c855b0f979c None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:45.083598 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 586/1145] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:57:44 2022] GET /identity/v3/projects/74ec3a9f55124b149ed71539ff22b8b4 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:45.089501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14829820-e277-4a8e-ab24-5d668c1e66fb None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:45.138089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:45.186194 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14829820-e277-4a8e-ab24-5d668c1e66fb None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:45.219215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a51d4e-d324-482e-a904-2f9f8592857d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:45.219804 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a51d4e-d324-482e-a904-2f9f8592857d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:45.220082 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a51d4e-d324-482e-a904-2f9f8592857d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:45.220350 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a51d4e-d324-482e-a904-2f9f8592857d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:45.221093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a51d4e-d324-482e-a904-2f9f8592857d None admin] RBAC: Adding query filter params (role.id=a0a76de72fba4036bc1e3e64df717c7f, scope.project.id=74ec3a9f55124b149ed71539ff22b8b4, user.id=faa05c33c1744c67bb596b44f9eb725b) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:45.221396 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a51d4e-d324-482e-a904-2f9f8592857d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:45.222802 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a51d4e-d324-482e-a904-2f9f8592857d None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:45.227980 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 560/1146] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:57:45 2022] GET /identity/v3/role_assignments?user.id=faa05c33c1744c67bb596b44f9eb725b&scope.project.id=74ec3a9f55124b149ed71539ff22b8b4&role.id=a0a76de72fba4036bc1e3e64df717c7f => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:47.066292 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67157233-2ce9-4692-a6bd-39ac1212a742 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:47.066823 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67157233-2ce9-4692-a6bd-39ac1212a742 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:47.066823 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67157233-2ce9-4692-a6bd-39ac1212a742 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:47.068201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 587/1147] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:47.073120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ded61-a284-416d-94d0-00882e8d68c9 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:47.073494 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ded61-a284-416d-94d0-00882e8d68c9 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:47.073662 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ded61-a284-416d-94d0-00882e8d68c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:47.091710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-2a8ded61-a284-416d-94d0-00882e8d68c9 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:47.092873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-2a8ded61-a284-416d-94d0-00882e8d68c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:47.155510 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 561/1148] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:47.432819 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67157233-2ce9-4692-a6bd-39ac1212a742 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:47.509489 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:47.532911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67157233-2ce9-4692-a6bd-39ac1212a742 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:47.566364 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3b420-d489-4c2b-acbb-be3b45a23d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:47.566975 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3b420-d489-4c2b-acbb-be3b45a23d71 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:47.567284 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3b420-d489-4c2b-acbb-be3b45a23d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:47.567559 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3b420-d489-4c2b-acbb-be3b45a23d71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:47.577342 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b3b420-d489-4c2b-acbb-be3b45a23d71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:57:47.582144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-81b3b420-d489-4c2b-acbb-be3b45a23d71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:57:47.582144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:47.582144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:47.582144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:47.582144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:47.582144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:47.582144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:47.582144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:47.582144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:47.582144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:47.582144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:47.582144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:47.582144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:57:47.582144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:57:47.582144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:57:47.582144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:57:47.583283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:57:47.583283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:57:47.583283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:57:47.583283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:57:47.583283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:47.583283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:47.583283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:47.583283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:47.583283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:47.583283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:47.583283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:47.583283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:47.583283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:47.583283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:47.583283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:47.583283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:47.584341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:47.584341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:47.584341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:47.584341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:47.584341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:57:47.584341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:57:47.584341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:57:47.584341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:57:47.584341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:57:47.584341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:57:47.584341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:57:47.584341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:47.584341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 588/1149] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:57:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:47.589108 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ded61-a284-416d-94d0-00882e8d68c9 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:47.638340 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:47.690472 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ded61-a284-416d-94d0-00882e8d68c9 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:47.722420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7791b0-289d-40fe-89ad-ffbd5d873104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:47.723018 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7791b0-289d-40fe-89ad-ffbd5d873104 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:47.723318 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7791b0-289d-40fe-89ad-ffbd5d873104 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:47.723586 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7791b0-289d-40fe-89ad-ffbd5d873104 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:47.724241 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7791b0-289d-40fe-89ad-ffbd5d873104 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:47.724663 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7791b0-289d-40fe-89ad-ffbd5d873104 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:47.726016 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7791b0-289d-40fe-89ad-ffbd5d873104 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:47.733634 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 562/1150] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:57:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:47.740030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3b420-d489-4c2b-acbb-be3b45a23d71 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:47.837001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3b420-d489-4c2b-acbb-be3b45a23d71 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:47.869445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67802454-d752-461b-a49b-e4ad28f21f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:47.870052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67802454-d752-461b-a49b-e4ad28f21f82 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:47.870332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67802454-d752-461b-a49b-e4ad28f21f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:47.870605 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67802454-d752-461b-a49b-e4ad28f21f82 None admin] PATH_INFO: `/v3/users/faa05c33c1744c67bb596b44f9eb725b` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:47.871728 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67802454-d752-461b-a49b-e4ad28f21f82 None admin] RBAC: Authorizing `identity:get_user(user_id=faa05c33c1744c67bb596b44f9eb725b)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:47.873146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67802454-d752-461b-a49b-e4ad28f21f82 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:47.874676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 589/1151] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:57:47 2022] GET /identity/v3/users/faa05c33c1744c67bb596b44f9eb725b => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:47.880761 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7791b0-289d-40fe-89ad-ffbd5d873104 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:47.983057 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7791b0-289d-40fe-89ad-ffbd5d873104 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:47.990043 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:48.015309 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac8b72-9fac-4d68-8126-12df0d7f8c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:48.015948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac8b72-9fac-4d68-8126-12df0d7f8c5e None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:48.016223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac8b72-9fac-4d68-8126-12df0d7f8c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:48.016489 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac8b72-9fac-4d68-8126-12df0d7f8c5e None admin] PATH_INFO: `/v3/projects/74ec3a9f55124b149ed71539ff22b8b4` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:48.017580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac8b72-9fac-4d68-8126-12df0d7f8c5e None admin] RBAC: Authorizing `identity:get_project(project_id=74ec3a9f55124b149ed71539ff22b8b4)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:48.018980 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eac8b72-9fac-4d68-8126-12df0d7f8c5e None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:48.020604 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 563/1152] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:57:47 2022] GET /identity/v3/projects/74ec3a9f55124b149ed71539ff22b8b4 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:48.026525 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67802454-d752-461b-a49b-e4ad28f21f82 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:48.030448 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:48.130291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67802454-d752-461b-a49b-e4ad28f21f82 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:48.164574 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21478f0-26d5-4289-a9d3-c9b13ba9464a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:48.165229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21478f0-26d5-4289-a9d3-c9b13ba9464a None admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:48.165508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21478f0-26d5-4289-a9d3-c9b13ba9464a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:48.165783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21478f0-26d5-4289-a9d3-c9b13ba9464a None admin] PATH_INFO: `/v3/projects/74ec3a9f55124b149ed71539ff22b8b4/users/faa05c33c1744c67bb596b44f9eb725b/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:48.167694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21478f0-26d5-4289-a9d3-c9b13ba9464a None admin] RBAC: Authorizing `identity:create_grant(project_id=74ec3a9f55124b149ed71539ff22b8b4, user_id=faa05c33c1744c67bb596b44f9eb725b, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:48.169122 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21478f0-26d5-4289-a9d3-c9b13ba9464a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:48.183954 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 590/1153] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:57:48 2022] PUT /identity/v3/projects/74ec3a9f55124b149ed71539ff22b8b4/users/faa05c33c1744c67bb596b44f9eb725b/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 08:57:50.067882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c32f0-f8e0-4afd-bed1-07af925b30ce None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:50.068528 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c32f0-f8e0-4afd-bed1-07af925b30ce None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:50.068528 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c32f0-f8e0-4afd-bed1-07af925b30ce None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:50.069741 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 564/1154] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:50.074766 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe032c5-1610-4f8b-9f2b-94e21f2caf61 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:50.075161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe032c5-1610-4f8b-9f2b-94e21f2caf61 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:50.075350 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe032c5-1610-4f8b-9f2b-94e21f2caf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:50.094180 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-9fe032c5-1610-4f8b-9f2b-94e21f2caf61 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:50.095410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-9fe032c5-1610-4f8b-9f2b-94e21f2caf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:50.151643 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:50.161492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 591/1155] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:50.431809 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c32f0-f8e0-4afd-bed1-07af925b30ce None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:50.529575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c32f0-f8e0-4afd-bed1-07af925b30ce None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:50.564554 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89bb27-c513-45db-b789-e589f24432b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:50.565167 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89bb27-c513-45db-b789-e589f24432b3 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:50.565444 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89bb27-c513-45db-b789-e589f24432b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:50.565711 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c89bb27-c513-45db-b789-e589f24432b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:50.577230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c89bb27-c513-45db-b789-e589f24432b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:57:50.582072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-3c89bb27-c513-45db-b789-e589f24432b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:57:50.582072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:50.582072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:50.582072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:50.582072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:50.582072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:50.582072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:50.582072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:50.582072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:50.582072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:50.582072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:50.582072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:50.582072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:57:50.582072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:57:50.582072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:57:50.582072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:57:50.583224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:57:50.583224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:57:50.583224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:57:50.583224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:57:50.583224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:50.583224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:50.583224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:50.583224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:50.583224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:50.583224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:50.583224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:50.583224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:50.583224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:50.583224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:50.583224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:50.583224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:50.584204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:50.584204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:50.584204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:50.584204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:50.584204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:57:50.584204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:57:50.584204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:57:50.584204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:57:50.584204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:57:50.584204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:57:50.584204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:57:50.584204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:57:50.584204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 565/1156] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:57:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:50.589284 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe032c5-1610-4f8b-9f2b-94e21f2caf61 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:50.684106 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe032c5-1610-4f8b-9f2b-94e21f2caf61 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:50.717836 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d901056-85c9-4954-be09-f199f3514748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:50.718402 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d901056-85c9-4954-be09-f199f3514748 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:50.718673 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d901056-85c9-4954-be09-f199f3514748 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:50.718916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d901056-85c9-4954-be09-f199f3514748 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:50.719596 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d901056-85c9-4954-be09-f199f3514748 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:50.719897 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d901056-85c9-4954-be09-f199f3514748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:50.721262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d901056-85c9-4954-be09-f199f3514748 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:50.729251 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 592/1157] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:57:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:50.735521 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89bb27-c513-45db-b789-e589f24432b3 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:50.768714 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:50.831329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c89bb27-c513-45db-b789-e589f24432b3 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:50.862789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade58f9c-95f9-4906-9ea4-e44d4277dc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:50.863417 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade58f9c-95f9-4906-9ea4-e44d4277dc2b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:50.863691 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade58f9c-95f9-4906-9ea4-e44d4277dc2b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:50.863959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade58f9c-95f9-4906-9ea4-e44d4277dc2b None admin] PATH_INFO: `/v3/users/faa05c33c1744c67bb596b44f9eb725b` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:50.865048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade58f9c-95f9-4906-9ea4-e44d4277dc2b None admin] RBAC: Authorizing `identity:get_user(user_id=faa05c33c1744c67bb596b44f9eb725b)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:50.866442 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade58f9c-95f9-4906-9ea4-e44d4277dc2b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:50.867993 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 566/1158] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:57:50 2022] GET /identity/v3/users/faa05c33c1744c67bb596b44f9eb725b => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:50.873713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d901056-85c9-4954-be09-f199f3514748 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:50.928473 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:50.974048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d901056-85c9-4954-be09-f199f3514748 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:51.011869 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbabc6-fbf0-4276-be38-09549a39a857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:51.012715 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbabc6-fbf0-4276-be38-09549a39a857 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:51.013117 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbabc6-fbf0-4276-be38-09549a39a857 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:51.013496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbabc6-fbf0-4276-be38-09549a39a857 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:51.077389 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbabc6-fbf0-4276-be38-09549a39a857 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:51.078814 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbabc6-fbf0-4276-be38-09549a39a857 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:51.130084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:51.172958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 593/1159] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:57:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 08:57:51.178853 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade58f9c-95f9-4906-9ea4-e44d4277dc2b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:51.273534 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade58f9c-95f9-4906-9ea4-e44d4277dc2b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:51.274921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:51.309336 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39381e8c-4a08-4d75-9e47-12ece271d73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:51.309979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39381e8c-4a08-4d75-9e47-12ece271d73d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:51.310259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39381e8c-4a08-4d75-9e47-12ece271d73d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:51.310534 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39381e8c-4a08-4d75-9e47-12ece271d73d None admin] PATH_INFO: `/v3/projects/74ec3a9f55124b149ed71539ff22b8b4` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:51.311690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39381e8c-4a08-4d75-9e47-12ece271d73d None admin] RBAC: Authorizing `identity:get_project(project_id=74ec3a9f55124b149ed71539ff22b8b4)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:51.313319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39381e8c-4a08-4d75-9e47-12ece271d73d None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:51.314981 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 567/1160] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:57:51 2022] GET /identity/v3/projects/74ec3a9f55124b149ed71539ff22b8b4 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:51.321450 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbabc6-fbf0-4276-be38-09549a39a857 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:51.419305 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbabc6-fbf0-4276-be38-09549a39a857 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:51.451719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180a283-f769-4795-8699-ffb05366e29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:51.452315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180a283-f769-4795-8699-ffb05366e29a None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:51.452595 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180a283-f769-4795-8699-ffb05366e29a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:51.453421 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180a283-f769-4795-8699-ffb05366e29a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:51.454283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180a283-f769-4795-8699-ffb05366e29a None admin] RBAC: Adding query filter params (role.id=a0a76de72fba4036bc1e3e64df717c7f, scope.project.id=74ec3a9f55124b149ed71539ff22b8b4, user.id=faa05c33c1744c67bb596b44f9eb725b) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:51.454591 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180a283-f769-4795-8699-ffb05366e29a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:51.456048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b180a283-f769-4795-8699-ffb05366e29a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:51.461707 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 594/1161] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:57:51 2022] GET /identity/v3/role_assignments?user.id=faa05c33c1744c67bb596b44f9eb725b&scope.project.id=74ec3a9f55124b149ed71539ff22b8b4&role.id=a0a76de72fba4036bc1e3e64df717c7f => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:53.372931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c022960-db3a-460d-b734-e11f6d896465 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:53.373373 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c022960-db3a-460d-b734-e11f6d896465 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:53.373497 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c022960-db3a-460d-b734-e11f6d896465 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:53.375454 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 568/1162] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:53.380463 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b104aeb-12d0-4b90-bf88-b374c534536f None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:53.380849 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b104aeb-12d0-4b90-bf88-b374c534536f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:53.381009 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b104aeb-12d0-4b90-bf88-b374c534536f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:53.400050 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-3b104aeb-12d0-4b90-bf88-b374c534536f None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:53.401228 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-3b104aeb-12d0-4b90-bf88-b374c534536f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:53.414895 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:53.466931 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 595/1163] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:53.733896 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c022960-db3a-460d-b734-e11f6d896465 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:53.832462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c022960-db3a-460d-b734-e11f6d896465 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:53.864296 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4919aae-d6f5-41df-bd5a-7afbc9037bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:53.864881 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4919aae-d6f5-41df-bd5a-7afbc9037bdd None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:53.865157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4919aae-d6f5-41df-bd5a-7afbc9037bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:53.865427 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4919aae-d6f5-41df-bd5a-7afbc9037bdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:53.902107 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:53.929671 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4919aae-d6f5-41df-bd5a-7afbc9037bdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:53.931126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4919aae-d6f5-41df-bd5a-7afbc9037bdd None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:54.024203 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 569/1164] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:57:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 08:57:54.029938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b104aeb-12d0-4b90-bf88-b374c534536f None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:54.127168 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b104aeb-12d0-4b90-bf88-b374c534536f None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:54.160289 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c464e6e-7c8e-4c8d-860f-c8da6ce178c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:54.160904 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c464e6e-7c8e-4c8d-860f-c8da6ce178c1 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:54.161186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c464e6e-7c8e-4c8d-860f-c8da6ce178c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:54.161459 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c464e6e-7c8e-4c8d-860f-c8da6ce178c1 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:54.168152 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c464e6e-7c8e-4c8d-860f-c8da6ce178c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:54.169586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c464e6e-7c8e-4c8d-860f-c8da6ce178c1 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:54.182109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-4c464e6e-7c8e-4c8d-860f-c8da6ce178c1 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 02 08:57:54.182109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:54.182109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 02 08:57:54.182109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 02 08:57:54.182109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:57:54.182109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:57:54.182109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:57:54.182109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:57:54.182109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 02 08:57:54.182109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:54.182109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:57:54.182109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:54.182109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:57:54.182109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:57:54.182109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:57:54.182109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:57:54.182109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:57:54.183599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:57:54.183599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:57:54.183599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:57:54.183599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:57:54.183599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:57:54.183599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:57:54.183599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:57:54.183599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:57:54.183599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:57:54.183599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:57:54.183599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:57:54.183599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:57:54.183599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:57:54.183599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:57:54.183599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:57:54.183599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:57:54.184647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:57:54.184647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:57:54.184647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:57:54.184647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:57:54.184647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:57:54.184647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:57:54.184647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:57:54.184647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:57:54.184647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:57:54.184647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:57:54.184647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 02 08:57:54.184647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:57:54.184647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 02 08:57:54.184647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:57:54.187242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 596/1165] 10.222.0.36 () {56 vars in 1117 bytes} [Fri Sep 2 08:57:54 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:57:54.189280 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4919aae-d6f5-41df-bd5a-7afbc9037bdd None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:54.254086 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:54.283354 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4919aae-d6f5-41df-bd5a-7afbc9037bdd None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:54.318083 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fdf531-a561-4f63-8fba-bca2398b838c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:54.318676 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fdf531-a561-4f63-8fba-bca2398b838c None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:54.318949 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fdf531-a561-4f63-8fba-bca2398b838c None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:54.319245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fdf531-a561-4f63-8fba-bca2398b838c None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:54.319874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fdf531-a561-4f63-8fba-bca2398b838c None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:57:54.320170 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fdf531-a561-4f63-8fba-bca2398b838c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:54.321511 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fdf531-a561-4f63-8fba-bca2398b838c None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:54.328886 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 570/1166] 10.222.0.36 () {56 vars in 1115 bytes} [Fri Sep 2 08:57:54 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:56.189816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804481b-7226-4c5a-8e5a-17b7fc3c26ff None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:56.190359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804481b-7226-4c5a-8e5a-17b7fc3c26ff None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:56.190359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804481b-7226-4c5a-8e5a-17b7fc3c26ff None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:56.191743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 597/1167] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:56.196787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612393c5-a92c-4e53-a515-b05f8f69b9c7 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:56.197168 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612393c5-a92c-4e53-a515-b05f8f69b9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:56.197327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612393c5-a92c-4e53-a515-b05f8f69b9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:56.216335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-612393c5-a92c-4e53-a515-b05f8f69b9c7 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:56.217501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-612393c5-a92c-4e53-a515-b05f8f69b9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:56.284200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 571/1168] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:56.548332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804481b-7226-4c5a-8e5a-17b7fc3c26ff None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:56.560780 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:56.646690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b804481b-7226-4c5a-8e5a-17b7fc3c26ff None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:56.678948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddfc054-031c-4eba-a0b3-beb17f4499a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:56.679566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfc054-031c-4eba-a0b3-beb17f4499a6 None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:56.679846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfc054-031c-4eba-a0b3-beb17f4499a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:56.680117 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddfc054-031c-4eba-a0b3-beb17f4499a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:56.680778 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddfc054-031c-4eba-a0b3-beb17f4499a6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:56.682126 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddfc054-031c-4eba-a0b3-beb17f4499a6 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:56.697794 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 598/1169] 10.222.0.36 () {60 vars in 1138 bytes} [Fri Sep 2 08:57:56 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:57:58.597915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9a5cf-e73d-4dea-989a-ea050e577a5f None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:58.598438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9a5cf-e73d-4dea-989a-ea050e577a5f None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:58.598438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b9a5cf-e73d-4dea-989a-ea050e577a5f None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:58.599839 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 572/1170] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:57:58.604942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8658f3f-ae5a-4fa3-bd23-739dffc37457 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:58.605373 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8658f3f-ae5a-4fa3-bd23-739dffc37457 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:58.605477 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8658f3f-ae5a-4fa3-bd23-739dffc37457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:58.623684 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-f8658f3f-ae5a-4fa3-bd23-739dffc37457 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:57:58.624848 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f8658f3f-ae5a-4fa3-bd23-739dffc37457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:57:58.684408 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:58.688986 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 599/1171] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:57:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:57:58.955659 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9a5cf-e73d-4dea-989a-ea050e577a5f None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:59.014664 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:57:59.051579 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b9a5cf-e73d-4dea-989a-ea050e577a5f None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:59.083734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86237107-9324-448c-8c6a-9a7704e5300d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:59.084336 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86237107-9324-448c-8c6a-9a7704e5300d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:59.084611 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86237107-9324-448c-8c6a-9a7704e5300d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:59.084879 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86237107-9324-448c-8c6a-9a7704e5300d None admin] PATH_INFO: `/v3/domains/e7b93662774b4c1e926e25c27ce3ea1a` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:59.095381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86237107-9324-448c-8c6a-9a7704e5300d None admin] RBAC: Authorizing `identity:get_domain(domain_id=e7b93662774b4c1e926e25c27ce3ea1a)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:59.096805 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86237107-9324-448c-8c6a-9a7704e5300d None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:59.098333 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 573/1172] 10.222.0.36 () {56 vars in 1183 bytes} [Fri Sep 2 08:57:58 2022] GET /identity/v3/domains/e7b93662774b4c1e926e25c27ce3ea1a => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:57:59.104100 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8658f3f-ae5a-4fa3-bd23-739dffc37457 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:57:59.198958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8658f3f-ae5a-4fa3-bd23-739dffc37457 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:57:59.231492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4d1c5-77aa-4874-9733-1eb315962946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:57:59.232000 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4d1c5-77aa-4874-9733-1eb315962946 None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:57:59.232279 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4d1c5-77aa-4874-9733-1eb315962946 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:57:59.232574 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4d1c5-77aa-4874-9733-1eb315962946 None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:57:59.233203 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4d1c5-77aa-4874-9733-1eb315962946 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:57:59.234642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4d1c5-77aa-4874-9733-1eb315962946 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:57:59.260224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 600/1173] 10.222.0.36 () {60 vars in 1141 bytes} [Fri Sep 2 08:57:59 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 08:58:01.158598 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f35815-beed-4c02-8822-78bd8e17a9c8 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:01.159151 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f35815-beed-4c02-8822-78bd8e17a9c8 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:01.159151 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f35815-beed-4c02-8822-78bd8e17a9c8 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:01.160499 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 574/1174] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:58:01.165488 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f668d24-303b-4051-8a39-ea24c31c08c8 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:01.165915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f668d24-303b-4051-8a39-ea24c31c08c8 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:01.166035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f668d24-303b-4051-8a39-ea24c31c08c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:01.184060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-0f668d24-303b-4051-8a39-ea24c31c08c8 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:58:01.185224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0f668d24-303b-4051-8a39-ea24c31c08c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:58:01.230792 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:58:01.250751 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 601/1175] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:58:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:58:01.522266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f35815-beed-4c02-8822-78bd8e17a9c8 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:58:01.622221 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f35815-beed-4c02-8822-78bd8e17a9c8 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:58:01.645027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:58:01.655508 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0a5fd-4c95-4c91-893c-21267ccdee2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:58:01.656151 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0a5fd-4c95-4c91-893c-21267ccdee2f None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:01.656431 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0a5fd-4c95-4c91-893c-21267ccdee2f None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:01.656703 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0a5fd-4c95-4c91-893c-21267ccdee2f None admin] PATH_INFO: `/v3/domains/e7b93662774b4c1e926e25c27ce3ea1a` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:01.657809 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0a5fd-4c95-4c91-893c-21267ccdee2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=e7b93662774b4c1e926e25c27ce3ea1a)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:58:01.659257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0a5fd-4c95-4c91-893c-21267ccdee2f None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:58:01.660807 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 575/1176] 10.222.0.36 () {56 vars in 1183 bytes} [Fri Sep 2 08:58:01 2022] GET /identity/v3/domains/e7b93662774b4c1e926e25c27ce3ea1a => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:58:01.666408 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f668d24-303b-4051-8a39-ea24c31c08c8 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:58:01.762115 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f668d24-303b-4051-8a39-ea24c31c08c8 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:58:01.794896 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b6548-ef1a-4446-9d9e-35b1e160fc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:58:01.795471 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b6548-ef1a-4446-9d9e-35b1e160fc15 None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:01.795751 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b6548-ef1a-4446-9d9e-35b1e160fc15 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:01.796020 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b6548-ef1a-4446-9d9e-35b1e160fc15 None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:01.797190 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b6548-ef1a-4446-9d9e-35b1e160fc15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:58:01.798602 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b6548-ef1a-4446-9d9e-35b1e160fc15 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:58:01.818609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 602/1177] 10.222.0.36 () {60 vars in 1132 bytes} [Fri Sep 2 08:58:01 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:58:03.741759 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33c5b2-0579-4111-b28c-c39b4636c2d1 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:03.742308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33c5b2-0579-4111-b28c-c39b4636c2d1 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:03.742308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33c5b2-0579-4111-b28c-c39b4636c2d1 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:03.743606 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 576/1178] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:58:03.748728 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c20173-53c0-4f87-8d82-d5a40a614c56 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:03.749153 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c20173-53c0-4f87-8d82-d5a40a614c56 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:03.749272 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c20173-53c0-4f87-8d82-d5a40a614c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:03.768068 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-f4c20173-53c0-4f87-8d82-d5a40a614c56 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:58:03.769220 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f4c20173-53c0-4f87-8d82-d5a40a614c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:58:03.797211 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:58:03.835761 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 603/1179] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:58:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:58:04.100375 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33c5b2-0579-4111-b28c-c39b4636c2d1 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:58:04.198303 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33c5b2-0579-4111-b28c-c39b4636c2d1 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:58:04.229212 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f683215-57ce-4f64-b53f-23bda8fbc2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:58:04.229816 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f683215-57ce-4f64-b53f-23bda8fbc2d3 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:04.230090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f683215-57ce-4f64-b53f-23bda8fbc2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:04.230353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f683215-57ce-4f64-b53f-23bda8fbc2d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:04.240858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f683215-57ce-4f64-b53f-23bda8fbc2d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:58:04.245578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-9f683215-57ce-4f64-b53f-23bda8fbc2d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:58:04.245578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:58:04.245578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:58:04.245578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:58:04.245578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:58:04.245578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:58:04.245578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:58:04.245578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:58:04.245578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:58:04.245578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:58:04.245578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:58:04.245578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:58:04.245578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:58:04.245578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:58:04.245578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:58:04.245578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:58:04.246738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:58:04.246738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:58:04.246738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:58:04.246738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:58:04.246738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:58:04.246738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:58:04.246738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:58:04.246738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:58:04.246738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:58:04.246738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:58:04.246738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:58:04.246738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:58:04.246738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:58:04.246738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:58:04.246738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:58:04.246738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:58:04.247880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:58:04.247880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:58:04.247880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:58:04.247880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:58:04.247880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:58:04.247880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:58:04.247880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:58:04.247880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:58:04.247880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:58:04.247880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:58:04.247880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:58:04.247880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:58:04.247880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 577/1180] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:58:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:58:04.253132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c20173-53c0-4f87-8d82-d5a40a614c56 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:58:04.347959 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c20173-53c0-4f87-8d82-d5a40a614c56 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:58:04.381283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d515df-fc48-49b8-9feb-edbdc2b752c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:58:04.381873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d515df-fc48-49b8-9feb-edbdc2b752c7 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:04.382152 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d515df-fc48-49b8-9feb-edbdc2b752c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:04.382417 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d515df-fc48-49b8-9feb-edbdc2b752c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:04.383063 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d515df-fc48-49b8-9feb-edbdc2b752c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:58:04.383387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d515df-fc48-49b8-9feb-edbdc2b752c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:58:04.384738 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d515df-fc48-49b8-9feb-edbdc2b752c7 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:58:04.392467 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 604/1181] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:58:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:58:04.398101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f683215-57ce-4f64-b53f-23bda8fbc2d3 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:58:04.448978 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:58:04.494534 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f683215-57ce-4f64-b53f-23bda8fbc2d3 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:58:04.525864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94ad93-c19f-41aa-a0e2-24a2dd0f8ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:58:04.526466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94ad93-c19f-41aa-a0e2-24a2dd0f8ffe None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:04.526742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94ad93-c19f-41aa-a0e2-24a2dd0f8ffe None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:04.527011 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94ad93-c19f-41aa-a0e2-24a2dd0f8ffe None admin] PATH_INFO: `/v3/users/52b60d1e2da540fca637493665095c0c` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:04.540015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94ad93-c19f-41aa-a0e2-24a2dd0f8ffe None admin] RBAC: Authorizing `identity:get_user(user_id=52b60d1e2da540fca637493665095c0c)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:58:04.541431 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94ad93-c19f-41aa-a0e2-24a2dd0f8ffe None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:58:04.542983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 578/1182] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:58:04 2022] GET /identity/v3/users/52b60d1e2da540fca637493665095c0c => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:58:04.549374 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d515df-fc48-49b8-9feb-edbdc2b752c7 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:58:04.579019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:58:04.646893 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d515df-fc48-49b8-9feb-edbdc2b752c7 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:58:04.678407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a36ca1-f64d-4be6-b69d-2dff84a560e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:58:04.678992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a36ca1-f64d-4be6-b69d-2dff84a560e3 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:04.679295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a36ca1-f64d-4be6-b69d-2dff84a560e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:04.679565 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a36ca1-f64d-4be6-b69d-2dff84a560e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:04.743767 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a36ca1-f64d-4be6-b69d-2dff84a560e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:58:04.745197 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a36ca1-f64d-4be6-b69d-2dff84a560e3 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:58:04.773152 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:58:04.840207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 605/1183] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:58:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 08:58:04.846650 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94ad93-c19f-41aa-a0e2-24a2dd0f8ffe None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:58:04.941801 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94ad93-c19f-41aa-a0e2-24a2dd0f8ffe None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:58:04.952235 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:58:04.975949 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecc0b4-2754-494c-8d4e-bcd5da381863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:58:04.976563 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecc0b4-2754-494c-8d4e-bcd5da381863 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:04.976842 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecc0b4-2754-494c-8d4e-bcd5da381863 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:04.977115 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecc0b4-2754-494c-8d4e-bcd5da381863 None admin] PATH_INFO: `/v3/projects/4196643f050d4255bb419fc2b004cd29` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:04.978218 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ecc0b4-2754-494c-8d4e-bcd5da381863 None admin] RBAC: Authorizing `identity:get_project(project_id=4196643f050d4255bb419fc2b004cd29)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:58:04.979669 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ecc0b4-2754-494c-8d4e-bcd5da381863 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:58:04.981275 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 579/1184] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:58:04 2022] GET /identity/v3/projects/4196643f050d4255bb419fc2b004cd29 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:58:04.987197 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a36ca1-f64d-4be6-b69d-2dff84a560e3 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:58:05.081914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a36ca1-f64d-4be6-b69d-2dff84a560e3 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:58:05.109598 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:58:05.115543 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0f1c1-55d8-4cb3-ba79-27f921ff77ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:58:05.116131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0f1c1-55d8-4cb3-ba79-27f921ff77ac None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:05.116406 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0f1c1-55d8-4cb3-ba79-27f921ff77ac None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:05.116672 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0f1c1-55d8-4cb3-ba79-27f921ff77ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:05.117383 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e0f1c1-55d8-4cb3-ba79-27f921ff77ac None admin] RBAC: Adding query filter params (role.id=a0a76de72fba4036bc1e3e64df717c7f, scope.project.id=4196643f050d4255bb419fc2b004cd29, user.id=52b60d1e2da540fca637493665095c0c) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:58:05.117685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e0f1c1-55d8-4cb3-ba79-27f921ff77ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:58:05.119120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e0f1c1-55d8-4cb3-ba79-27f921ff77ac None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:58:05.124322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 606/1185] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:58:04 2022] GET /identity/v3/role_assignments?user.id=52b60d1e2da540fca637493665095c0c&scope.project.id=4196643f050d4255bb419fc2b004cd29&role.id=a0a76de72fba4036bc1e3e64df717c7f => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:58:06.991452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb023db-648d-4e6a-8609-15c093c91d6f None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:06.991981 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb023db-648d-4e6a-8609-15c093c91d6f None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:06.991981 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb023db-648d-4e6a-8609-15c093c91d6f None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:06.993320 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 580/1186] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:58:06.998540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762ca31-ad01-4c1c-a35e-5c9f328209fa None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:06.998932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762ca31-ad01-4c1c-a35e-5c9f328209fa None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:06.999110 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1762ca31-ad01-4c1c-a35e-5c9f328209fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:07.016621 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-1762ca31-ad01-4c1c-a35e-5c9f328209fa None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:58:07.017768 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-1762ca31-ad01-4c1c-a35e-5c9f328209fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:58:07.085489 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 607/1187] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:58:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:58:07.353616 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb023db-648d-4e6a-8609-15c093c91d6f None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:58:07.454188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb023db-648d-4e6a-8609-15c093c91d6f None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:58:07.486318 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a79de-2d96-47c4-b4ae-f59125d0a800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:58:07.486931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a79de-2d96-47c4-b4ae-f59125d0a800 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:07.487235 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a79de-2d96-47c4-b4ae-f59125d0a800 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:07.487505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a79de-2d96-47c4-b4ae-f59125d0a800 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:07.497428 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a79de-2d96-47c4-b4ae-f59125d0a800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:58:07.502696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-697a79de-2d96-47c4-b4ae-f59125d0a800 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:58:07.502696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:58:07.502696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:58:07.502696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:58:07.502696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:58:07.502696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:58:07.502696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:58:07.502696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:58:07.502696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:58:07.502696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:58:07.502696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:58:07.502696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:58:07.502696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:58:07.502696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:58:07.502696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:58:07.502696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:58:07.503837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:58:07.503837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:58:07.503837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:58:07.503837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:58:07.503837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:58:07.503837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:58:07.503837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:58:07.503837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:58:07.503837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:58:07.503837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:58:07.503837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:58:07.503837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:58:07.503837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:58:07.503837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:58:07.503837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:58:07.503837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:58:07.504891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:58:07.504891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:58:07.504891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:58:07.504891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:58:07.504891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:58:07.504891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:58:07.504891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:58:07.504891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:58:07.504891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:58:07.504891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:58:07.504891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:58:07.504891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:58:07.504891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 581/1188] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:58:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:58:07.509807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762ca31-ad01-4c1c-a35e-5c9f328209fa None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:58:07.605325 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1762ca31-ad01-4c1c-a35e-5c9f328209fa None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:58:07.627779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:58:07.638533 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b136f8b-3eac-4a61-9129-6c9103f840e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:58:07.639164 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b136f8b-3eac-4a61-9129-6c9103f840e8 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:07.639444 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b136f8b-3eac-4a61-9129-6c9103f840e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:07.639711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b136f8b-3eac-4a61-9129-6c9103f840e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:07.640892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b136f8b-3eac-4a61-9129-6c9103f840e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:58:07.641187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b136f8b-3eac-4a61-9129-6c9103f840e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:58:07.642538 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b136f8b-3eac-4a61-9129-6c9103f840e8 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:58:07.650298 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 608/1189] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:58:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:58:07.656062 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a79de-2d96-47c4-b4ae-f59125d0a800 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:58:07.694867 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:58:07.753764 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a79de-2d96-47c4-b4ae-f59125d0a800 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:58:07.786275 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c9def-76e9-449d-9175-9e6eee6f493b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:58:07.786885 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c9def-76e9-449d-9175-9e6eee6f493b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:07.787189 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c9def-76e9-449d-9175-9e6eee6f493b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:07.787466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515c9def-76e9-449d-9175-9e6eee6f493b None admin] PATH_INFO: `/v3/users/52b60d1e2da540fca637493665095c0c` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:07.788573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c9def-76e9-449d-9175-9e6eee6f493b None admin] RBAC: Authorizing `identity:get_user(user_id=52b60d1e2da540fca637493665095c0c)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:58:07.789994 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515c9def-76e9-449d-9175-9e6eee6f493b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:58:07.791571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 582/1190] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:58:07 2022] GET /identity/v3/users/52b60d1e2da540fca637493665095c0c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:58:07.797665 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b136f8b-3eac-4a61-9129-6c9103f840e8 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:58:07.892825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b136f8b-3eac-4a61-9129-6c9103f840e8 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:58:07.924326 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3f7e4-21ee-4fe8-b6a4-cd914d2c0ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:58:07.925018 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3f7e4-21ee-4fe8-b6a4-cd914d2c0ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:07.925296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3f7e4-21ee-4fe8-b6a4-cd914d2c0ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:07.925565 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3f7e4-21ee-4fe8-b6a4-cd914d2c0ac7 None admin] PATH_INFO: `/v3/projects/4196643f050d4255bb419fc2b004cd29` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:07.926641 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3f7e4-21ee-4fe8-b6a4-cd914d2c0ac7 None admin] RBAC: Authorizing `identity:get_project(project_id=4196643f050d4255bb419fc2b004cd29)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:58:07.928070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3f7e4-21ee-4fe8-b6a4-cd914d2c0ac7 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:58:07.929629 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 609/1191] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:58:07 2022] GET /identity/v3/projects/4196643f050d4255bb419fc2b004cd29 => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:58:07.935793 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c9def-76e9-449d-9175-9e6eee6f493b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:58:08.031018 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515c9def-76e9-449d-9175-9e6eee6f493b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:58:08.037965 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:58:08.065096 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad139e8-d597-4dc2-b689-7ec3adbf0e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:58:08.065731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad139e8-d597-4dc2-b689-7ec3adbf0e59 None admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:08.066009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad139e8-d597-4dc2-b689-7ec3adbf0e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:08.066282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad139e8-d597-4dc2-b689-7ec3adbf0e59 None admin] PATH_INFO: `/v3/projects/4196643f050d4255bb419fc2b004cd29/users/52b60d1e2da540fca637493665095c0c/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:08.068178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad139e8-d597-4dc2-b689-7ec3adbf0e59 None admin] RBAC: Authorizing `identity:create_grant(project_id=4196643f050d4255bb419fc2b004cd29, user_id=52b60d1e2da540fca637493665095c0c, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:58:08.069582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad139e8-d597-4dc2-b689-7ec3adbf0e59 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:58:08.082184 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 583/1192] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:58:07 2022] PUT /identity/v3/projects/4196643f050d4255bb419fc2b004cd29/users/52b60d1e2da540fca637493665095c0c/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:58:09.948131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cb055-078d-4745-b91d-2f2f2211a9fb None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:09.948654 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cb055-078d-4745-b91d-2f2f2211a9fb None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:09.948654 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cb055-078d-4745-b91d-2f2f2211a9fb None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:09.949998 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 610/1193] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:58:09.955085 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbacda79-2fe4-496f-9458-a8a095c27053 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:09.955475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbacda79-2fe4-496f-9458-a8a095c27053 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:09.955665 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbacda79-2fe4-496f-9458-a8a095c27053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:09.974654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-fbacda79-2fe4-496f-9458-a8a095c27053 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:58:09.975874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-fbacda79-2fe4-496f-9458-a8a095c27053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:58:10.038451 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 584/1194] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:58:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:58:10.303401 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32cb055-078d-4745-b91d-2f2f2211a9fb None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:58:10.358509 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:58:10.402538 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32cb055-078d-4745-b91d-2f2f2211a9fb None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:58:10.434702 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34092482-9727-48e1-91f7-7ea3d1ce3587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:58:10.435343 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34092482-9727-48e1-91f7-7ea3d1ce3587 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:10.435617 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34092482-9727-48e1-91f7-7ea3d1ce3587 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:10.435882 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34092482-9727-48e1-91f7-7ea3d1ce3587 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:10.446411 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34092482-9727-48e1-91f7-7ea3d1ce3587 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:58:10.451138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-34092482-9727-48e1-91f7-7ea3d1ce3587 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:58:10.451138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:58:10.451138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:58:10.451138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:58:10.451138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:58:10.451138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:58:10.451138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:58:10.451138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:58:10.451138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:58:10.451138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:58:10.451138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:58:10.451138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:58:10.451138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:58:10.451138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:58:10.451138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:58:10.451138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:58:10.452261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:58:10.452261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:58:10.452261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:58:10.452261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:58:10.452261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:58:10.452261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:58:10.452261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:58:10.452261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:58:10.452261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:58:10.452261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:58:10.452261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:58:10.452261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:58:10.452261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:58:10.452261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:58:10.452261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:58:10.452261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:58:10.453690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:58:10.453690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:58:10.453690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:58:10.453690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:58:10.453690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:58:10.453690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:58:10.453690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:58:10.453690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:58:10.453690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:58:10.453690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:58:10.453690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:58:10.453690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:58:10.453690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 611/1195] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:58:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:58:10.458764 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbacda79-2fe4-496f-9458-a8a095c27053 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:58:10.547031 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:58:10.553089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbacda79-2fe4-496f-9458-a8a095c27053 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:58:10.587298 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553cec8-3824-4acd-abb8-e0220aca5a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:58:10.587837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553cec8-3824-4acd-abb8-e0220aca5a09 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:10.588110 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553cec8-3824-4acd-abb8-e0220aca5a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:10.588376 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553cec8-3824-4acd-abb8-e0220aca5a09 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:10.589037 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1553cec8-3824-4acd-abb8-e0220aca5a09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:58:10.589334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1553cec8-3824-4acd-abb8-e0220aca5a09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:58:10.590668 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1553cec8-3824-4acd-abb8-e0220aca5a09 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:58:10.598330 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 585/1196] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:58:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:58:10.603921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34092482-9727-48e1-91f7-7ea3d1ce3587 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:58:10.699436 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34092482-9727-48e1-91f7-7ea3d1ce3587 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:58:10.712808 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:58:10.735023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312771c-d36a-43cd-9ee9-5f1bad760718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:58:10.735666 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312771c-d36a-43cd-9ee9-5f1bad760718 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:10.735941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312771c-d36a-43cd-9ee9-5f1bad760718 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:10.736210 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312771c-d36a-43cd-9ee9-5f1bad760718 None admin] PATH_INFO: `/v3/users/52b60d1e2da540fca637493665095c0c` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:10.737292 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f312771c-d36a-43cd-9ee9-5f1bad760718 None admin] RBAC: Authorizing `identity:get_user(user_id=52b60d1e2da540fca637493665095c0c)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:58:10.738731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f312771c-d36a-43cd-9ee9-5f1bad760718 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:58:10.740321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 612/1197] 10.222.0.36 () {56 vars in 1177 bytes} [Fri Sep 2 08:58:10 2022] GET /identity/v3/users/52b60d1e2da540fca637493665095c0c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:58:10.746097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553cec8-3824-4acd-abb8-e0220aca5a09 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:58:10.841937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553cec8-3824-4acd-abb8-e0220aca5a09 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:58:10.873571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44d281-45ad-449b-ad89-eb1ef5f7b847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:58:10.874155 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44d281-45ad-449b-ad89-eb1ef5f7b847 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:10.874963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44d281-45ad-449b-ad89-eb1ef5f7b847 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:10.875258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44d281-45ad-449b-ad89-eb1ef5f7b847 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:10.900163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:58:10.939823 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44d281-45ad-449b-ad89-eb1ef5f7b847 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:58:10.941237 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44d281-45ad-449b-ad89-eb1ef5f7b847 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:58:11.033067 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 586/1198] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:58:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 08:58:11.038783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312771c-d36a-43cd-9ee9-5f1bad760718 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:58:11.135385 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312771c-d36a-43cd-9ee9-5f1bad760718 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:58:11.167130 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb109cd-20a6-46be-bc2c-4193b9ef3fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:58:11.167740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb109cd-20a6-46be-bc2c-4193b9ef3fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:11.168020 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb109cd-20a6-46be-bc2c-4193b9ef3fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:11.168292 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb109cd-20a6-46be-bc2c-4193b9ef3fd6 None admin] PATH_INFO: `/v3/projects/4196643f050d4255bb419fc2b004cd29` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:11.169392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb109cd-20a6-46be-bc2c-4193b9ef3fd6 None admin] RBAC: Authorizing `identity:get_project(project_id=4196643f050d4255bb419fc2b004cd29)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:58:11.170810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb109cd-20a6-46be-bc2c-4193b9ef3fd6 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:58:11.172916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 613/1199] 10.222.0.36 () {56 vars in 1186 bytes} [Fri Sep 2 08:58:11 2022] GET /identity/v3/projects/4196643f050d4255bb419fc2b004cd29 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:58:11.178721 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44d281-45ad-449b-ad89-eb1ef5f7b847 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:58:11.234723 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:58:11.273559 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44d281-45ad-449b-ad89-eb1ef5f7b847 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:58:11.306807 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8dad9-3c86-414d-83b9-36fb6480a10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:58:11.307421 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8dad9-3c86-414d-83b9-36fb6480a10d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:58:11.307699 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8dad9-3c86-414d-83b9-36fb6480a10d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:58:11.307970 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8dad9-3c86-414d-83b9-36fb6480a10d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:58:11.308939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8dad9-3c86-414d-83b9-36fb6480a10d None admin] RBAC: Adding query filter params (role.id=a0a76de72fba4036bc1e3e64df717c7f, scope.project.id=4196643f050d4255bb419fc2b004cd29, user.id=52b60d1e2da540fca637493665095c0c) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:58:11.309243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8dad9-3c86-414d-83b9-36fb6480a10d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:58:11.310643 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8dad9-3c86-414d-83b9-36fb6480a10d None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:58:11.316161 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 587/1200] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:58:11 2022] GET /identity/v3/role_assignments?user.id=52b60d1e2da540fca637493665095c0c&scope.project.id=4196643f050d4255bb419fc2b004cd29&role.id=a0a76de72fba4036bc1e3e64df717c7f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:58:21.163060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=75 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:58:21.313637 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=39 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:06.228505 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf7665-8cc0-4eae-96cd-c20b5954ee22 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:06.229088 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf7665-8cc0-4eae-96cd-c20b5954ee22 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:06.229088 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf7665-8cc0-4eae-96cd-c20b5954ee22 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:06.230831 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 614/1201] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 08:59:06.237894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8da52a-c994-48a4-b85c-2283df19eeae None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:06.238382 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8da52a-c994-48a4-b85c-2283df19eeae None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:06.238468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8da52a-c994-48a4-b85c-2283df19eeae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:06.258268 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-dd8da52a-c994-48a4-b85c-2283df19eeae None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:59:06.259541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-dd8da52a-c994-48a4-b85c-2283df19eeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:59:06.347789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 588/1202] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:59:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:59:06.650128 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf7665-8cc0-4eae-96cd-c20b5954ee22 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:06.776328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf7665-8cc0-4eae-96cd-c20b5954ee22 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:06.825265 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a308598-9bf3-452c-9a0b-e7b291e44d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:06.825931 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a308598-9bf3-452c-9a0b-e7b291e44d6d None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:06.826243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a308598-9bf3-452c-9a0b-e7b291e44d6d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:06.826502 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a308598-9bf3-452c-9a0b-e7b291e44d6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:06.839740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a308598-9bf3-452c-9a0b-e7b291e44d6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:06.841275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a308598-9bf3-452c-9a0b-e7b291e44d6d None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:06.859642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-6a308598-9bf3-452c-9a0b-e7b291e44d6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:06.859642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:06.859642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:06.859642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:06.859642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:06.859642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:06.859642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:06.859642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:06.859642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:06.859642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:06.859642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:06.859642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:06.859642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:59:06.859642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:59:06.859642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:59:06.859642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:59:06.860772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:06.860772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:06.860772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:06.860772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:06.860772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:06.860772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:06.860772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:06.860772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:06.860772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:06.860772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:06.860772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:06.860772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:06.860772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:06.860772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:06.860772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:06.860772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:06.860772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:59:06.864593 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:59:06.864593 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:06.864593 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:59:06.864593 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 615/1203] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:59:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:06.867876 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8da52a-c994-48a4-b85c-2283df19eeae None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:07.002464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8da52a-c994-48a4-b85c-2283df19eeae None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:07.044555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:07.053083 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ab925-b99b-47b9-a33a-799a27ed14dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:07.053770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ab925-b99b-47b9-a33a-799a27ed14dc None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:07.054198 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ab925-b99b-47b9-a33a-799a27ed14dc None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:07.054493 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ab925-b99b-47b9-a33a-799a27ed14dc None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:07.055379 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0ab925-b99b-47b9-a33a-799a27ed14dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:07.055693 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0ab925-b99b-47b9-a33a-799a27ed14dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:07.057312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0ab925-b99b-47b9-a33a-799a27ed14dc None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:07.071784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 589/1204] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:07.078010 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a308598-9bf3-452c-9a0b-e7b291e44d6d None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:07.147583 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:07.207254 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a308598-9bf3-452c-9a0b-e7b291e44d6d None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:07.247385 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178db3b3-920b-48dc-b4b9-dd62d5e43cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:07.248356 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178db3b3-920b-48dc-b4b9-dd62d5e43cd6 None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:07.248480 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178db3b3-920b-48dc-b4b9-dd62d5e43cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:07.248787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178db3b3-920b-48dc-b4b9-dd62d5e43cd6 None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:07.249703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178db3b3-920b-48dc-b4b9-dd62d5e43cd6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:07.251868 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178db3b3-920b-48dc-b4b9-dd62d5e43cd6 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:07.288685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 616/1205] 10.222.0.36 () {60 vars in 1132 bytes} [Fri Sep 2 08:59:07 2022] POST /identity/v3/users => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:59:09.428264 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1aea9-d304-4963-bf00-42e2e8d70d31 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:09.428848 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1aea9-d304-4963-bf00-42e2e8d70d31 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:09.428960 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1aea9-d304-4963-bf00-42e2e8d70d31 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:09.430355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 590/1206] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:59:09.440589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac9af5-3fa0-436b-9dec-2d87af1e1364 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:09.441053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac9af5-3fa0-436b-9dec-2d87af1e1364 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:09.441143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eac9af5-3fa0-436b-9dec-2d87af1e1364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:09.460068 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-6eac9af5-3fa0-436b-9dec-2d87af1e1364 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:59:09.461370 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6eac9af5-3fa0-436b-9dec-2d87af1e1364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:59:09.535364 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 617/1207] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:59:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:59:09.802658 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1aea9-d304-4963-bf00-42e2e8d70d31 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:09.921138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1aea9-d304-4963-bf00-42e2e8d70d31 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:09.967858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c521d-5efa-4190-bb30-24bb9a94a6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:09.968519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c521d-5efa-4190-bb30-24bb9a94a6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:09.968803 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c521d-5efa-4190-bb30-24bb9a94a6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:09.969061 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c521d-5efa-4190-bb30-24bb9a94a6c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:09.979795 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c521d-5efa-4190-bb30-24bb9a94a6c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:59:09.985288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-745c521d-5efa-4190-bb30-24bb9a94a6c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:59:09.985288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:09.985288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:09.985288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:09.985288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:09.985288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:09.985288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:09.985288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:09.985288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:09.985288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:09.985288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:09.985288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:09.985288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:59:09.985288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:59:09.985288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:59:09.985288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:59:09.986283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:59:09.986283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:59:09.986283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:59:09.986283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:59:09.986283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:09.986283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:09.986283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:09.986283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:09.986283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:09.986283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:09.986283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:09.986283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:09.986283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:09.986283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:09.986283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:09.986283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:09.987834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:09.987834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:09.987834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:09.987834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:09.987834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:59:09.987834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:59:09.987834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:59:09.987834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:59:09.987834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:59:09.987834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:59:09.987834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:59:09.987834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:09.987834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 591/1208] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:59:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:09.992353 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac9af5-3fa0-436b-9dec-2d87af1e1364 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:10.036789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:10.105489 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eac9af5-3fa0-436b-9dec-2d87af1e1364 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:10.146522 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388a6c5-a7bb-4e98-bb73-961e8d6f9a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:10.147242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388a6c5-a7bb-4e98-bb73-961e8d6f9a6c None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:10.147524 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388a6c5-a7bb-4e98-bb73-961e8d6f9a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:10.147829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b388a6c5-a7bb-4e98-bb73-961e8d6f9a6c None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:10.148697 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388a6c5-a7bb-4e98-bb73-961e8d6f9a6c None admin] RBAC: Adding query filter params (name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:10.149032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388a6c5-a7bb-4e98-bb73-961e8d6f9a6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:10.150625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b388a6c5-a7bb-4e98-bb73-961e8d6f9a6c None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:10.160295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 618/1209] 10.222.0.36 () {56 vars in 1103 bytes} [Fri Sep 2 08:59:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:59:10.166196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c521d-5efa-4190-bb30-24bb9a94a6c2 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:10.228164 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:10.285150 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c521d-5efa-4190-bb30-24bb9a94a6c2 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:10.325193 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42477396-4635-45d5-bb65-1acb26f0f353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:10.325926 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42477396-4635-45d5-bb65-1acb26f0f353 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:10.326278 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42477396-4635-45d5-bb65-1acb26f0f353 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:10.326780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42477396-4635-45d5-bb65-1acb26f0f353 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:10.342970 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42477396-4635-45d5-bb65-1acb26f0f353 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:10.344668 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42477396-4635-45d5-bb65-1acb26f0f353 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:10.362770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-42477396-4635-45d5-bb65-1acb26f0f353 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:10.362770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:10.362770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:10.362770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:10.362770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:10.362770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:10.362770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:10.362770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:10.362770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:10.362770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:10.362770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:10.362770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:10.362770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:59:10.362770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:59:10.362770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:59:10.362770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:59:10.364010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:10.364010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:10.364010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:10.364010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:10.364010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:10.364010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:10.364010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:10.364010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:10.364010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:10.364010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:10.364010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:10.364010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:10.364010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:10.364010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:10.364010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:10.364010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:10.364010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:59:10.364890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:59:10.364890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:10.364890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:10.364890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 592/1210] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:59:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:10.370768 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388a6c5-a7bb-4e98-bb73-961e8d6f9a6c None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:10.497997 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:10.502415 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b388a6c5-a7bb-4e98-bb73-961e8d6f9a6c None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:10.542431 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3798f96-39b1-4412-a388-70e8323fe071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:10.543042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3798f96-39b1-4412-a388-70e8323fe071 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:10.543420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3798f96-39b1-4412-a388-70e8323fe071 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:10.543788 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3798f96-39b1-4412-a388-70e8323fe071 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:10.544458 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3798f96-39b1-4412-a388-70e8323fe071 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:10.544843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3798f96-39b1-4412-a388-70e8323fe071 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:10.547023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3798f96-39b1-4412-a388-70e8323fe071 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:10.562327 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 619/1211] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:10.568474 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42477396-4635-45d5-bb65-1acb26f0f353 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:10.684801 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42477396-4635-45d5-bb65-1acb26f0f353 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:10.686507 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:10.722967 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25285ea9-7449-4c47-b665-e0a593a24381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:10.723634 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25285ea9-7449-4c47-b665-e0a593a24381 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:10.723923 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25285ea9-7449-4c47-b665-e0a593a24381 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:10.724213 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25285ea9-7449-4c47-b665-e0a593a24381 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:10.733954 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25285ea9-7449-4c47-b665-e0a593a24381 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:10.735819 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25285ea9-7449-4c47-b665-e0a593a24381 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:10.751042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-25285ea9-7449-4c47-b665-e0a593a24381 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 02 08:59:10.751042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:10.751042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:10.751042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:10.751042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:10.751042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:10.751042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:10.751042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:10.751042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:10.751042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:10.751042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:10.751042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:10.751042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:59:10.751042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:59:10.751042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:59:10.751042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:59:10.752455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:10.752455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:10.752455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:59:10.752455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:59:10.752455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:59:10.752455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:59:10.752455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:10.752455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:10.752455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:10.752455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:10.752455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:10.752455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:10.752455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:10.752455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:10.752455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:10.752455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:10.752455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:10.752455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:10.753549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:10.753549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:10.753549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:59:10.753549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:59:10.753549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:59:10.753549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:59:10.753549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:59:10.753549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:59:10.753549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 02 08:59:10.753549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:10.753549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 593/1212] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:59:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:59:10.758889 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3798f96-39b1-4412-a388-70e8323fe071 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:10.876141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3798f96-39b1-4412-a388-70e8323fe071 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:10.917530 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c64f7a-588d-435b-8352-d57dd6d77852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:10.918122 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c64f7a-588d-435b-8352-d57dd6d77852 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:10.918437 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c64f7a-588d-435b-8352-d57dd6d77852 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:10.918654 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c64f7a-588d-435b-8352-d57dd6d77852 None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:10.919415 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c64f7a-588d-435b-8352-d57dd6d77852 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:10.919749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c64f7a-588d-435b-8352-d57dd6d77852 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:10.921218 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c64f7a-588d-435b-8352-d57dd6d77852 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:10.941961 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 620/1213] 10.222.0.36 () {56 vars in 1143 bytes} [Fri Sep 2 08:59:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:10.948497 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25285ea9-7449-4c47-b665-e0a593a24381 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:11.066359 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25285ea9-7449-4c47-b665-e0a593a24381 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:11.106399 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d9bad-7656-460c-9349-f146705a1af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:11.107011 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d9bad-7656-460c-9349-f146705a1af7 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:11.107348 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d9bad-7656-460c-9349-f146705a1af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:11.107627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d9bad-7656-460c-9349-f146705a1af7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:11.140688 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:11.185170 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d9bad-7656-460c-9349-f146705a1af7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:11.187197 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d9bad-7656-460c-9349-f146705a1af7 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:11.311949 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 594/1214] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:59:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 08:59:11.317943 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c64f7a-588d-435b-8352-d57dd6d77852 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:11.340864 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:11.447765 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c64f7a-588d-435b-8352-d57dd6d77852 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:11.485313 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da1c79-1f8f-4ae3-9b1c-f318eb87c579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:11.486147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da1c79-1f8f-4ae3-9b1c-f318eb87c579 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:11.486517 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da1c79-1f8f-4ae3-9b1c-f318eb87c579 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:11.486880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da1c79-1f8f-4ae3-9b1c-f318eb87c579 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:11.499489 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da1c79-1f8f-4ae3-9b1c-f318eb87c579 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:11.501026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da1c79-1f8f-4ae3-9b1c-f318eb87c579 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:11.517187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-06da1c79-1f8f-4ae3-9b1c-f318eb87c579 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:11.517187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:11.517187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:11.517187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:11.517187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:11.517187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:11.517187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:11.517187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:11.517187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:11.517187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:11.517187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:11.517187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:11.517187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:59:11.517187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:59:11.517187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:59:11.517187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:59:11.518177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:11.518177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:11.518177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:11.518177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:11.518177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:11.518177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:11.518177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:11.518177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:11.518177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:11.518177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:11.518177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:11.518177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:11.518177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:11.518177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:11.518177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:11.518177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:11.518177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:59:11.519114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:59:11.519114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:11.519114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:59:11.519114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 621/1215] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:59:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:11.524042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d9bad-7656-460c-9349-f146705a1af7 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:11.597890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:11.641724 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d9bad-7656-460c-9349-f146705a1af7 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:11.680013 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6e856-7c03-4a33-bbea-90e38826459f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:11.680754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6e856-7c03-4a33-bbea-90e38826459f None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:11.681080 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6e856-7c03-4a33-bbea-90e38826459f None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:11.681436 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6e856-7c03-4a33-bbea-90e38826459f None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:11.682185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6e856-7c03-4a33-bbea-90e38826459f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:11.682478 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6e856-7c03-4a33-bbea-90e38826459f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:11.684127 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6e856-7c03-4a33-bbea-90e38826459f None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:11.697802 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 595/1216] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:59:11.704013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da1c79-1f8f-4ae3-9b1c-f318eb87c579 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:11.794477 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:11.832391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da1c79-1f8f-4ae3-9b1c-f318eb87c579 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:11.872611 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063214f-1f6a-47d9-ab2e-eaeb133b315d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:11.873407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063214f-1f6a-47d9-ab2e-eaeb133b315d None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:11.873725 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063214f-1f6a-47d9-ab2e-eaeb133b315d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:11.874026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063214f-1f6a-47d9-ab2e-eaeb133b315d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:11.881353 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063214f-1f6a-47d9-ab2e-eaeb133b315d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:11.882807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063214f-1f6a-47d9-ab2e-eaeb133b315d None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:11.894854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-6063214f-1f6a-47d9-ab2e-eaeb133b315d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:59:11.894854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:11.894854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:11.894854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:11.894854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:11.894854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:11.894854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:11.894854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:11.894854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:11.894854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:11.894854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:11.894854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:11.894854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:59:11.894854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:59:11.894854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:59:11.896012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:59:11.896012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:11.896012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:11.896012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:11.896012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:11.896012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:11.896012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:11.896012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:11.896012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:11.896012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:11.896012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:11.896012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:11.896012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:11.896012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:11.896012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:11.896012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:11.896012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:11.897141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:59:11.897141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:59:11.897141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:59:11.897141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:59:11.897141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:59:11.897141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:59:11.897141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:59:11.897141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:59:11.897141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 622/1217] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:59:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:11.901699 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6e856-7c03-4a33-bbea-90e38826459f None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:12.004856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6e856-7c03-4a33-bbea-90e38826459f None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:12.032209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:12.046000 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae71a64f-f2da-4a93-897f-4424a6e0c08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:12.046612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71a64f-f2da-4a93-897f-4424a6e0c08f None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:12.047984 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71a64f-f2da-4a93-897f-4424a6e0c08f None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:12.048268 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71a64f-f2da-4a93-897f-4424a6e0c08f None admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:12.048970 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae71a64f-f2da-4a93-897f-4424a6e0c08f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:12.049279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae71a64f-f2da-4a93-897f-4424a6e0c08f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:12.050697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae71a64f-f2da-4a93-897f-4424a6e0c08f None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:12.069916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 596/1218] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:59:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:12.070981 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063214f-1f6a-47d9-ab2e-eaeb133b315d None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:12.189894 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063214f-1f6a-47d9-ab2e-eaeb133b315d None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:12.221888 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:12.229183 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491b76b-9cc0-4129-ad7e-746f301d0607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:12.229991 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491b76b-9cc0-4129-ad7e-746f301d0607 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:12.230370 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491b76b-9cc0-4129-ad7e-746f301d0607 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:12.230736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491b76b-9cc0-4129-ad7e-746f301d0607 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:12.231602 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491b76b-9cc0-4129-ad7e-746f301d0607 None admin] RBAC: Adding query filter params (role.id=637ed3cd3d484839ab21de02c2bbe293, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=6a0f0fdbbd20492d8c35ec4b73404632) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:12.232011 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491b76b-9cc0-4129-ad7e-746f301d0607 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:12.234098 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491b76b-9cc0-4129-ad7e-746f301d0607 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:12.251663 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 623/1219] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:59:12 2022] GET /identity/v3/role_assignments?user.id=6a0f0fdbbd20492d8c35ec4b73404632&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=637ed3cd3d484839ab21de02c2bbe293 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:59:14.233317 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae66074-b681-4160-a401-d3955ae43ed6 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:14.233889 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae66074-b681-4160-a401-d3955ae43ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:14.234247 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae66074-b681-4160-a401-d3955ae43ed6 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:14.237307 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 597/1220] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:59:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:59:14.242629 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cd595-14fa-4946-8ca4-7f8c79b0daa0 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:14.243192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cd595-14fa-4946-8ca4-7f8c79b0daa0 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:14.243192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432cd595-14fa-4946-8ca4-7f8c79b0daa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:14.263629 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-432cd595-14fa-4946-8ca4-7f8c79b0daa0 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:59:14.264805 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-432cd595-14fa-4946-8ca4-7f8c79b0daa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:59:14.343976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 624/1221] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:59:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:59:14.631370 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae66074-b681-4160-a401-d3955ae43ed6 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:14.747812 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae66074-b681-4160-a401-d3955ae43ed6 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:14.784036 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e071d-2bd8-4019-aeee-0c9f0979e7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:14.784773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e071d-2bd8-4019-aeee-0c9f0979e7bb None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:14.785052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e071d-2bd8-4019-aeee-0c9f0979e7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:14.785316 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e071d-2bd8-4019-aeee-0c9f0979e7bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:14.796240 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07e071d-2bd8-4019-aeee-0c9f0979e7bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:59:14.801718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-a07e071d-2bd8-4019-aeee-0c9f0979e7bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:59:14.801718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:14.801718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:14.801718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:14.801718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:14.801718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:14.801718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:14.801718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:14.801718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:14.801718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:14.801718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:14.801718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:14.801718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:59:14.801718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:59:14.801718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:59:14.801718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:59:14.802755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:59:14.802755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:59:14.802755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:59:14.802755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:59:14.802755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:14.802755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:14.802755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:14.802755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:14.802755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:14.802755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:14.802755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:14.802755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:14.802755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:14.802755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:14.802755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:14.802755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:14.803629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:14.803629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:14.803629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:14.803629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:14.803629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:59:14.803629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:59:14.803629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:59:14.803629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:59:14.803629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:59:14.803629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:59:14.803629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:59:14.803629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:14.803629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 598/1222] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:59:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:14.809699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432cd595-14fa-4946-8ca4-7f8c79b0daa0 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:14.922969 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432cd595-14fa-4946-8ca4-7f8c79b0daa0 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:14.948428 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:14.961395 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458495d-7b9e-4531-aa3b-e08337fac90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:14.962006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458495d-7b9e-4531-aa3b-e08337fac90d None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:14.962323 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458495d-7b9e-4531-aa3b-e08337fac90d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:14.962596 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458495d-7b9e-4531-aa3b-e08337fac90d None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:14.963296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7458495d-7b9e-4531-aa3b-e08337fac90d None admin] RBAC: Adding query filter params (name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:14.963631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7458495d-7b9e-4531-aa3b-e08337fac90d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:14.965274 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7458495d-7b9e-4531-aa3b-e08337fac90d None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:14.974476 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 625/1223] 10.222.0.36 () {56 vars in 1103 bytes} [Fri Sep 2 08:59:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:14.980992 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e071d-2bd8-4019-aeee-0c9f0979e7bb None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:15.026531 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:15.086211 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07e071d-2bd8-4019-aeee-0c9f0979e7bb None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:15.124230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e8bde-2ee5-47bb-b307-59bbfe724092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:15.124929 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e8bde-2ee5-47bb-b307-59bbfe724092 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:15.125212 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e8bde-2ee5-47bb-b307-59bbfe724092 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:15.125486 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e8bde-2ee5-47bb-b307-59bbfe724092 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:15.136429 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e8bde-2ee5-47bb-b307-59bbfe724092 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:15.138059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e8bde-2ee5-47bb-b307-59bbfe724092 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:15.153582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-050e8bde-2ee5-47bb-b307-59bbfe724092 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:15.153582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:15.153582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:15.153582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:15.153582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:15.153582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:15.153582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:15.153582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:15.153582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:15.153582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:15.153582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:15.153582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:15.153582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:59:15.153582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:59:15.153582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:59:15.153582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:59:15.155223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:15.155223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:15.155223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:15.155223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:15.155223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:15.155223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:15.155223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:15.155223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:15.155223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:15.155223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:15.155223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:15.155223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:15.155223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:15.155223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:15.155223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:15.155223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:15.155223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:59:15.156330 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:59:15.156330 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:15.156330 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:15.159226 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 599/1224] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:59:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:15.161192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458495d-7b9e-4531-aa3b-e08337fac90d None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:15.265443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458495d-7b9e-4531-aa3b-e08337fac90d None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:15.301223 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2afe9-a7a1-421e-9e6b-35f21f1b685c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:15.301888 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2afe9-a7a1-421e-9e6b-35f21f1b685c None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:15.302140 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2afe9-a7a1-421e-9e6b-35f21f1b685c None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:15.302401 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2afe9-a7a1-421e-9e6b-35f21f1b685c None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:15.303025 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2afe9-a7a1-421e-9e6b-35f21f1b685c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:15.303356 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2afe9-a7a1-421e-9e6b-35f21f1b685c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:15.304718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2afe9-a7a1-421e-9e6b-35f21f1b685c None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:15.318816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 626/1225] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:15.325279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e8bde-2ee5-47bb-b307-59bbfe724092 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:15.427098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:15.435286 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e8bde-2ee5-47bb-b307-59bbfe724092 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:15.475980 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b54761-345c-4a91-a2c1-d6f81311c893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:15.476602 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b54761-345c-4a91-a2c1-d6f81311c893 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:15.476877 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b54761-345c-4a91-a2c1-d6f81311c893 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:15.477146 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b54761-345c-4a91-a2c1-d6f81311c893 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:15.484947 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b54761-345c-4a91-a2c1-d6f81311c893 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:15.486370 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b54761-345c-4a91-a2c1-d6f81311c893 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:15.497488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-f6b54761-345c-4a91-a2c1-d6f81311c893 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 02 08:59:15.497488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:15.497488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:15.497488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:15.497488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:15.497488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:15.497488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:15.497488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:15.497488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:15.497488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:15.497488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:15.497488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:15.497488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:59:15.497488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:59:15.497488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:59:15.497488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:59:15.499433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:15.499433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:15.499433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:59:15.499433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:59:15.499433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:59:15.499433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:59:15.499433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:15.499433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:15.499433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:15.499433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:15.499433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:15.499433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:15.499433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:15.499433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:15.499433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:15.499433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:15.499433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:15.499433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:15.501399 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:15.501399 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:15.501399 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:59:15.501399 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:59:15.501399 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:59:15.501399 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:59:15.501399 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:59:15.501399 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:59:15.501399 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 02 08:59:15.501399 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:15.501399 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 600/1226] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:59:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:15.506253 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2afe9-a7a1-421e-9e6b-35f21f1b685c None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:15.556335 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:15.616078 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2afe9-a7a1-421e-9e6b-35f21f1b685c None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:15.653178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad432b2f-686e-4dfa-9f80-2f87c301db5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:15.653963 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad432b2f-686e-4dfa-9f80-2f87c301db5a None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:15.655013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad432b2f-686e-4dfa-9f80-2f87c301db5a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:15.655368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad432b2f-686e-4dfa-9f80-2f87c301db5a None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:15.656378 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad432b2f-686e-4dfa-9f80-2f87c301db5a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:15.656688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad432b2f-686e-4dfa-9f80-2f87c301db5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:15.658531 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad432b2f-686e-4dfa-9f80-2f87c301db5a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:15.677855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 627/1227] 10.222.0.36 () {56 vars in 1143 bytes} [Fri Sep 2 08:59:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:15.684028 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b54761-345c-4a91-a2c1-d6f81311c893 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:15.805317 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b54761-345c-4a91-a2c1-d6f81311c893 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:15.851283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec677e-59a2-41be-99ab-a11c47b53e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:15.852232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec677e-59a2-41be-99ab-a11c47b53e43 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:15.852691 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec677e-59a2-41be-99ab-a11c47b53e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:15.852984 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec677e-59a2-41be-99ab-a11c47b53e43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:15.868514 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec677e-59a2-41be-99ab-a11c47b53e43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:15.870538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec677e-59a2-41be-99ab-a11c47b53e43 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:15.886063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-07ec677e-59a2-41be-99ab-a11c47b53e43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:15.886063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:15.886063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:15.886063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:15.886063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:15.886063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:15.886063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:15.886063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:15.886063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:15.886063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:15.886063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:15.886063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:15.886063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:59:15.886063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:59:15.886063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:59:15.886063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:59:15.887368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:15.887368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:15.887368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:15.887368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:15.887368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:15.887368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:15.887368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:15.887368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:15.887368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:15.887368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:15.887368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:15.887368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:15.887368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:15.887368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:15.887368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:15.887368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:15.887368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:59:15.888536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:59:15.888536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:15.888536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:15.892472 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 601/1228] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:59:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:15.893354 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad432b2f-686e-4dfa-9f80-2f87c301db5a None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:15.996505 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad432b2f-686e-4dfa-9f80-2f87c301db5a None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:15.998844 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:16.039057 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67e376-908d-4600-a72b-7e6af7bebbd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:16.039704 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67e376-908d-4600-a72b-7e6af7bebbd6 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:16.039995 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67e376-908d-4600-a72b-7e6af7bebbd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:16.040272 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67e376-908d-4600-a72b-7e6af7bebbd6 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:16.040988 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67e376-908d-4600-a72b-7e6af7bebbd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:16.041306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67e376-908d-4600-a72b-7e6af7bebbd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:16.042699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67e376-908d-4600-a72b-7e6af7bebbd6 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:16.055420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 628/1229] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:59:16.061706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec677e-59a2-41be-99ab-a11c47b53e43 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:16.072353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:16.171467 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec677e-59a2-41be-99ab-a11c47b53e43 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:16.211084 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d237d-0e1e-45e9-a6f1-0c729b58ed98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:16.211715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d237d-0e1e-45e9-a6f1-0c729b58ed98 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:16.211990 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d237d-0e1e-45e9-a6f1-0c729b58ed98 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:16.212260 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d237d-0e1e-45e9-a6f1-0c729b58ed98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:16.218786 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d237d-0e1e-45e9-a6f1-0c729b58ed98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:16.220239 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d237d-0e1e-45e9-a6f1-0c729b58ed98 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:16.234445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-192d237d-0e1e-45e9-a6f1-0c729b58ed98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:59:16.234445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:16.234445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:16.234445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:16.234445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:16.234445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:16.234445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:16.234445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:16.234445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:16.234445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:16.234445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:16.234445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:16.234445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:59:16.234445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:59:16.234445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:59:16.235630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:59:16.235630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:16.235630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:16.235630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:16.235630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:16.235630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:16.235630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:16.235630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:16.235630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:16.235630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:16.235630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:16.235630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:16.235630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:16.235630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:16.235630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:16.235630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:16.235630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:16.236748 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:59:16.236748 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:59:16.236748 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:59:16.236748 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:59:16.236748 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:59:16.236748 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:59:16.236748 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:59:16.236748 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:16.236748 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 602/1230] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:59:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:16.241647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67e376-908d-4600-a72b-7e6af7bebbd6 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:16.350836 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67e376-908d-4600-a72b-7e6af7bebbd6 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:16.391869 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb2ae0-cbb7-46bd-b7cd-6aea565863eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:16.392663 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb2ae0-cbb7-46bd-b7cd-6aea565863eb None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:16.392975 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb2ae0-cbb7-46bd-b7cd-6aea565863eb None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:16.393252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb2ae0-cbb7-46bd-b7cd-6aea565863eb None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:16.394045 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb2ae0-cbb7-46bd-b7cd-6aea565863eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:16.394389 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb2ae0-cbb7-46bd-b7cd-6aea565863eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:16.395955 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb2ae0-cbb7-46bd-b7cd-6aea565863eb None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:16.411933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 629/1231] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:59:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:16.417958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d237d-0e1e-45e9-a6f1-0c729b58ed98 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:16.493479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:16.532285 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d237d-0e1e-45e9-a6f1-0c729b58ed98 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:16.573061 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df85b5-a532-4ef1-b27f-696c2fab57b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:16.573745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df85b5-a532-4ef1-b27f-696c2fab57b7 None admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:16.574028 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df85b5-a532-4ef1-b27f-696c2fab57b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:16.574306 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df85b5-a532-4ef1-b27f-696c2fab57b7 None admin] PATH_INFO: `/v3/projects/eef80493db09437b88f2b147a59928c1/users/6a0f0fdbbd20492d8c35ec4b73404632/roles/637ed3cd3d484839ab21de02c2bbe293` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:16.593432 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df85b5-a532-4ef1-b27f-696c2fab57b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=eef80493db09437b88f2b147a59928c1, user_id=6a0f0fdbbd20492d8c35ec4b73404632, role_id=637ed3cd3d484839ab21de02c2bbe293)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:16.595482 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84df85b5-a532-4ef1-b27f-696c2fab57b7 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:16.609393 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 603/1232] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:59:16 2022] PUT /identity/v3/projects/eef80493db09437b88f2b147a59928c1/users/6a0f0fdbbd20492d8c35ec4b73404632/roles/637ed3cd3d484839ab21de02c2bbe293 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:59:18.580354 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33943707-835a-45e3-97b8-54e3f7770493 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:18.580923 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33943707-835a-45e3-97b8-54e3f7770493 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:18.580923 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33943707-835a-45e3-97b8-54e3f7770493 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:18.582212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 630/1233] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:59:18.588271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8fb13-4bd0-42a9-a7e1-d13b21dc768c None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:18.588764 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8fb13-4bd0-42a9-a7e1-d13b21dc768c None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:18.588924 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8fb13-4bd0-42a9-a7e1-d13b21dc768c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:18.609697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-fcd8fb13-4bd0-42a9-a7e1-d13b21dc768c None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:59:18.611075 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-fcd8fb13-4bd0-42a9-a7e1-d13b21dc768c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:59:18.685198 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 604/1234] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:59:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:59:19.000036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33943707-835a-45e3-97b8-54e3f7770493 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:19.026851 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:19.105921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33943707-835a-45e3-97b8-54e3f7770493 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:19.141457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e1f98-8a59-492e-a6b8-1f9ca01c4de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:19.142307 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e1f98-8a59-492e-a6b8-1f9ca01c4de7 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:19.142589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e1f98-8a59-492e-a6b8-1f9ca01c4de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:19.142862 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e1f98-8a59-492e-a6b8-1f9ca01c4de7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:19.154882 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e1f98-8a59-492e-a6b8-1f9ca01c4de7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:59:19.159696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-205e1f98-8a59-492e-a6b8-1f9ca01c4de7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:59:19.159696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:19.159696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:19.159696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:19.159696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:19.159696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:19.159696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:19.159696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:19.159696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:19.159696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:19.159696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:19.159696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:19.159696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:59:19.159696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:59:19.159696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:59:19.159696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:59:19.160855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:59:19.160855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:59:19.160855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:59:19.160855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:59:19.160855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:19.160855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:19.160855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:19.160855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:19.160855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:19.160855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:19.160855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:19.160855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:19.160855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:19.160855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:19.160855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:19.160855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:19.161906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:19.161906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:19.161906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:19.161906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:19.161906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:59:19.161906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:59:19.161906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:59:19.161906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:59:19.161906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:59:19.161906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:59:19.161906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 08:59:19.161906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:59:19.161906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 631/1235] 10.222.0.36 () {56 vars in 1102 bytes} [Fri Sep 2 08:59:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:19.169400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8fb13-4bd0-42a9-a7e1-d13b21dc768c None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:19.216694 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:19.275141 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8fb13-4bd0-42a9-a7e1-d13b21dc768c None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:19.312549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838c0a9-a28d-4cad-b070-47d18cdb8d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:19.313154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838c0a9-a28d-4cad-b070-47d18cdb8d61 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:19.313431 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838c0a9-a28d-4cad-b070-47d18cdb8d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:19.313699 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838c0a9-a28d-4cad-b070-47d18cdb8d61 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:19.314379 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838c0a9-a28d-4cad-b070-47d18cdb8d61 None admin] RBAC: Adding query filter params (name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:19.314680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838c0a9-a28d-4cad-b070-47d18cdb8d61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:19.316221 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838c0a9-a28d-4cad-b070-47d18cdb8d61 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:19.324725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 605/1236] 10.222.0.36 () {56 vars in 1103 bytes} [Fri Sep 2 08:59:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:19.331249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e1f98-8a59-492e-a6b8-1f9ca01c4de7 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:19.421557 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:19.439846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e1f98-8a59-492e-a6b8-1f9ca01c4de7 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:19.476642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1053c9-3053-4e12-b21a-1b0e6bf1431d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:19.477282 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1053c9-3053-4e12-b21a-1b0e6bf1431d None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:19.477561 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1053c9-3053-4e12-b21a-1b0e6bf1431d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:19.477833 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1053c9-3053-4e12-b21a-1b0e6bf1431d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:19.488792 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1053c9-3053-4e12-b21a-1b0e6bf1431d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:19.490253 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1053c9-3053-4e12-b21a-1b0e6bf1431d None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:19.505540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-9f1053c9-3053-4e12-b21a-1b0e6bf1431d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:19.505540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:19.505540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:19.505540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:19.505540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:19.505540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:19.505540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:19.505540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:19.505540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:19.505540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:19.505540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:19.505540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:19.505540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:59:19.505540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:59:19.505540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:59:19.505540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:59:19.506711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:19.506711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:19.506711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:19.506711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:19.506711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:19.506711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:19.506711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:19.506711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:19.506711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:19.506711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:19.506711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:19.506711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:19.506711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:19.506711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:19.506711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:19.506711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:19.506711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:59:19.507682 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:59:19.507682 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:19.507682 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:59:19.507682 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 632/1237] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:59:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:19.512989 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838c0a9-a28d-4cad-b070-47d18cdb8d61 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:19.621082 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838c0a9-a28d-4cad-b070-47d18cdb8d61 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:19.632910 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:19.660164 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a7450-a20f-47d4-b484-ea92c158867f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:19.660905 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a7450-a20f-47d4-b484-ea92c158867f None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:19.661276 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a7450-a20f-47d4-b484-ea92c158867f None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:19.661648 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a7450-a20f-47d4-b484-ea92c158867f None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:19.662380 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a7450-a20f-47d4-b484-ea92c158867f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:19.662776 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a7450-a20f-47d4-b484-ea92c158867f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:19.664259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722a7450-a20f-47d4-b484-ea92c158867f None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:19.676260 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 606/1238] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:19.682231 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1053c9-3053-4e12-b21a-1b0e6bf1431d None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:19.794410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1053c9-3053-4e12-b21a-1b0e6bf1431d None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:19.830536 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fe4e3-8407-4364-bfa0-3cbda82913f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:19.831216 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fe4e3-8407-4364-bfa0-3cbda82913f0 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:19.831537 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fe4e3-8407-4364-bfa0-3cbda82913f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:19.831859 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fe4e3-8407-4364-bfa0-3cbda82913f0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:19.834828 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:19.841159 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959fe4e3-8407-4364-bfa0-3cbda82913f0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:19.842892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959fe4e3-8407-4364-bfa0-3cbda82913f0 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:19.854910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-959fe4e3-8407-4364-bfa0-3cbda82913f0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 02 08:59:19.854910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:19.854910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:19.854910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:19.854910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:19.854910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:19.854910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:19.854910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:19.854910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:19.854910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:19.854910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:19.854910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:19.854910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:59:19.854910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:59:19.854910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:59:19.854910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:59:19.856136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:19.856136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:19.856136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:59:19.856136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:59:19.856136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:59:19.856136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:59:19.856136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:19.856136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:19.856136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:19.856136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:19.856136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:19.856136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:19.856136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:19.856136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:19.856136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:19.856136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:19.856136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:19.856136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:19.857814 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:19.857814 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:19.857814 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:59:19.857814 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:59:19.857814 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:59:19.857814 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:59:19.857814 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:59:19.857814 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:59:19.857814 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 02 08:59:19.857814 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:59:19.859755 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 633/1239] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:59:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:19.865682 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a7450-a20f-47d4-b484-ea92c158867f None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:19.966977 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a7450-a20f-47d4-b484-ea92c158867f None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:20.001426 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63f168-39d2-4301-a034-aaa8069ef7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:20.002024 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63f168-39d2-4301-a034-aaa8069ef7ed None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:20.002299 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63f168-39d2-4301-a034-aaa8069ef7ed None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:20.002567 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63f168-39d2-4301-a034-aaa8069ef7ed None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:20.003323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63f168-39d2-4301-a034-aaa8069ef7ed None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:20.003622 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63f168-39d2-4301-a034-aaa8069ef7ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:20.005034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63f168-39d2-4301-a034-aaa8069ef7ed None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:20.026375 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 607/1240] 10.222.0.36 () {56 vars in 1143 bytes} [Fri Sep 2 08:59:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:59:20.032827 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fe4e3-8407-4364-bfa0-3cbda82913f0 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:20.141952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fe4e3-8407-4364-bfa0-3cbda82913f0 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:20.180392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5b44c-ef83-4c71-8f21-1ab7c74cee5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:20.181000 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5b44c-ef83-4c71-8f21-1ab7c74cee5a None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:20.181283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5b44c-ef83-4c71-8f21-1ab7c74cee5a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:20.181556 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5b44c-ef83-4c71-8f21-1ab7c74cee5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:20.253776 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:20.261586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e5b44c-ef83-4c71-8f21-1ab7c74cee5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:20.263170 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e5b44c-ef83-4c71-8f21-1ab7c74cee5a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:20.378768 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 634/1241] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:59:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 08:59:20.385232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63f168-39d2-4301-a034-aaa8069ef7ed None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:20.414216 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:20.488939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63f168-39d2-4301-a034-aaa8069ef7ed None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:20.528149 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef867b0-1072-4b92-96ab-66d26ee157b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:20.528897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef867b0-1072-4b92-96ab-66d26ee157b5 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:20.529239 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef867b0-1072-4b92-96ab-66d26ee157b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:20.529517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef867b0-1072-4b92-96ab-66d26ee157b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:20.539962 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef867b0-1072-4b92-96ab-66d26ee157b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:20.541397 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef867b0-1072-4b92-96ab-66d26ee157b5 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:20.556452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-8ef867b0-1072-4b92-96ab-66d26ee157b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:20.556452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:20.556452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:20.556452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:20.556452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:20.556452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:20.556452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:20.556452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:20.556452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:20.556452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:20.556452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:20.556452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:20.556452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:59:20.556452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:59:20.556452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:59:20.556452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:59:20.557617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:20.557617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:20.557617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:20.557617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:20.557617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:20.557617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:20.557617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:20.557617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:20.557617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:20.557617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:20.557617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:20.557617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:20.557617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:20.557617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:20.557617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:20.557617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:20.557617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:59:20.558510 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:59:20.558510 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:20.558510 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:20.558510 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 608/1242] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:20.566234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5b44c-ef83-4c71-8f21-1ab7c74cee5a None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:20.666298 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:20.673023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5b44c-ef83-4c71-8f21-1ab7c74cee5a None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:20.706466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1398f83-1624-4459-953c-7d71da88ded9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:20.707131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1398f83-1624-4459-953c-7d71da88ded9 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:20.707349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1398f83-1624-4459-953c-7d71da88ded9 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:20.707621 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1398f83-1624-4459-953c-7d71da88ded9 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:20.708241 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1398f83-1624-4459-953c-7d71da88ded9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:20.708536 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1398f83-1624-4459-953c-7d71da88ded9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:20.709894 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1398f83-1624-4459-953c-7d71da88ded9 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:20.721336 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 635/1243] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:20.727398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef867b0-1072-4b92-96ab-66d26ee157b5 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:20.822229 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:20.846257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef867b0-1072-4b92-96ab-66d26ee157b5 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:20.886049 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b80d58-d69b-46a9-bdf2-5d7a65174077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:20.886763 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b80d58-d69b-46a9-bdf2-5d7a65174077 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:20.887138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b80d58-d69b-46a9-bdf2-5d7a65174077 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:20.887591 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b80d58-d69b-46a9-bdf2-5d7a65174077 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:20.895560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b80d58-d69b-46a9-bdf2-5d7a65174077 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:20.897882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b80d58-d69b-46a9-bdf2-5d7a65174077 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:20.913130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-41b80d58-d69b-46a9-bdf2-5d7a65174077 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:59:20.913130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:20.913130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:20.913130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:20.913130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:20.913130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:20.913130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:20.913130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:20.913130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:20.913130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:20.913130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:20.913130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:20.913130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:59:20.913130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:59:20.913130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:59:20.914126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:59:20.914126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:20.914126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:20.914126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:20.914126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:20.914126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:20.914126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:20.914126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:20.914126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:20.914126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:20.914126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:20.914126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:20.914126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:20.914126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:20.914126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:20.914126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:20.914126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:20.915046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:59:20.915046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:59:20.915046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:59:20.915046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:59:20.915046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:59:20.915046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:59:20.915046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:59:20.915046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:20.915046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 609/1244] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:59:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:20.922548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1398f83-1624-4459-953c-7d71da88ded9 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:21.033264 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1398f83-1624-4459-953c-7d71da88ded9 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:21.072197 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f8f487-cbee-41ba-884f-d31c48ce66f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:21.072853 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8f487-cbee-41ba-884f-d31c48ce66f2 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:21.073081 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8f487-cbee-41ba-884f-d31c48ce66f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:21.073345 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f8f487-cbee-41ba-884f-d31c48ce66f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:21.074019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8f487-cbee-41ba-884f-d31c48ce66f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:21.074323 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8f487-cbee-41ba-884f-d31c48ce66f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:21.075838 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f8f487-cbee-41ba-884f-d31c48ce66f2 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:21.087902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 636/1245] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:59:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:21.094493 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b80d58-d69b-46a9-bdf2-5d7a65174077 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:21.208459 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b80d58-d69b-46a9-bdf2-5d7a65174077 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:21.244070 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a136eb5d-9f87-4685-b6c3-2747e5820c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:21.244647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136eb5d-9f87-4685-b6c3-2747e5820c2b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:21.244924 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136eb5d-9f87-4685-b6c3-2747e5820c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:21.245234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a136eb5d-9f87-4685-b6c3-2747e5820c2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:21.245933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136eb5d-9f87-4685-b6c3-2747e5820c2b None admin] RBAC: Adding query filter params (role.id=637ed3cd3d484839ab21de02c2bbe293, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=6a0f0fdbbd20492d8c35ec4b73404632) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:21.246233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136eb5d-9f87-4685-b6c3-2747e5820c2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:21.247717 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a136eb5d-9f87-4685-b6c3-2747e5820c2b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:21.253784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 610/1246] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:59:21 2022] GET /identity/v3/role_assignments?user.id=6a0f0fdbbd20492d8c35ec4b73404632&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=637ed3cd3d484839ab21de02c2bbe293 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:23.286089 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80106f97-088a-4e27-a33b-5f3ca4abb322 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:23.286705 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80106f97-088a-4e27-a33b-5f3ca4abb322 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:23.286705 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80106f97-088a-4e27-a33b-5f3ca4abb322 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:23.287981 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 637/1247] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:59:23.293315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed225b58-a78a-4ed3-a3c4-cdba13c6e121 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:23.293749 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed225b58-a78a-4ed3-a3c4-cdba13c6e121 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:23.293870 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed225b58-a78a-4ed3-a3c4-cdba13c6e121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:23.300186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:23.315404 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-ed225b58-a78a-4ed3-a3c4-cdba13c6e121 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:59:23.316777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ed225b58-a78a-4ed3-a3c4-cdba13c6e121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:59:23.416161 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 611/1248] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:59:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:59:23.683249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80106f97-088a-4e27-a33b-5f3ca4abb322 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:23.703771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:23.814435 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80106f97-088a-4e27-a33b-5f3ca4abb322 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:23.859383 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73124b-ef53-4ea7-90a1-7102e4764b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:23.860024 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73124b-ef53-4ea7-90a1-7102e4764b77 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:23.860306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73124b-ef53-4ea7-90a1-7102e4764b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:23.860565 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73124b-ef53-4ea7-90a1-7102e4764b77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:23.871112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73124b-ef53-4ea7-90a1-7102e4764b77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:59:23.876121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-9c73124b-ef53-4ea7-90a1-7102e4764b77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:59:23.876121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:23.876121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:23.876121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:23.876121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:23.876121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:23.876121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:23.876121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:23.876121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:23.876121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:23.876121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:23.876121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:23.876121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:59:23.876121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:59:23.876121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:59:23.876121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:59:23.878079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:59:23.878079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:59:23.878079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:59:23.878079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:59:23.878079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:23.878079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:23.878079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:23.878079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:23.878079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:23.878079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:23.878079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:23.878079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:23.878079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:23.878079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:23.878079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:23.878079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:23.880222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:23.880222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:23.880222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:23.880222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:23.880222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:59:23.880222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:59:23.880222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:59:23.880222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:59:23.880222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:59:23.880222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:59:23.880222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:59:23.880222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:59:23.880222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 638/1249] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:59:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:59:23.883258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed225b58-a78a-4ed3-a3c4-cdba13c6e121 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:23.987443 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed225b58-a78a-4ed3-a3c4-cdba13c6e121 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:24.017614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:24.023111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0609580-fe4a-461c-873d-069bfe8df0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:24.023747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0609580-fe4a-461c-873d-069bfe8df0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:24.024003 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0609580-fe4a-461c-873d-069bfe8df0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:24.024272 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0609580-fe4a-461c-873d-069bfe8df0a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:24.024934 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0609580-fe4a-461c-873d-069bfe8df0a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:24.025248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0609580-fe4a-461c-873d-069bfe8df0a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:24.026583 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0609580-fe4a-461c-873d-069bfe8df0a4 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:24.040535 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 612/1250] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:59:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:24.041818 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73124b-ef53-4ea7-90a1-7102e4764b77 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:24.137172 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:24.163014 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73124b-ef53-4ea7-90a1-7102e4764b77 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:24.200501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dee8e-9d55-4680-ac6b-8c8c270bcc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:24.201718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dee8e-9d55-4680-ac6b-8c8c270bcc3a None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:24.202049 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dee8e-9d55-4680-ac6b-8c8c270bcc3a None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:24.202367 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dee8e-9d55-4680-ac6b-8c8c270bcc3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:24.214813 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5dee8e-9d55-4680-ac6b-8c8c270bcc3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:24.216550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5dee8e-9d55-4680-ac6b-8c8c270bcc3a None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:24.233735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-2f5dee8e-9d55-4680-ac6b-8c8c270bcc3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:24.233735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:24.233735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:24.233735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:24.233735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:24.233735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:24.233735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:24.233735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:24.233735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:24.233735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:24.233735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:24.233735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:24.233735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:59:24.233735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:59:24.233735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:59:24.233735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:59:24.234902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:24.234902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:24.234902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:24.234902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:24.234902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:24.234902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:24.234902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:24.234902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:24.234902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:24.234902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:24.234902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:24.234902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:24.234902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:24.234902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:24.234902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:24.234902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:24.234902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:59:24.235889 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:59:24.235889 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:24.235889 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:59:24.235889 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 639/1251] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:59:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:24.241158 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0609580-fe4a-461c-873d-069bfe8df0a4 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:24.347093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0609580-fe4a-461c-873d-069bfe8df0a4 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:24.382866 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d8581-8095-4f3c-b1c4-bad4b8398403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:24.383586 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d8581-8095-4f3c-b1c4-bad4b8398403 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:24.383898 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d8581-8095-4f3c-b1c4-bad4b8398403 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:24.384214 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d8581-8095-4f3c-b1c4-bad4b8398403 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:24.384969 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d8581-8095-4f3c-b1c4-bad4b8398403 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:24.385315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d8581-8095-4f3c-b1c4-bad4b8398403 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:24.386869 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d8581-8095-4f3c-b1c4-bad4b8398403 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:24.399901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 613/1252] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:24.406117 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dee8e-9d55-4680-ac6b-8c8c270bcc3a None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:24.516794 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dee8e-9d55-4680-ac6b-8c8c270bcc3a None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:24.543766 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:24.552656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5fd3a4-f2dc-43d5-b8bb-8705693cf71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:24.553288 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5fd3a4-f2dc-43d5-b8bb-8705693cf71b None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:24.553583 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5fd3a4-f2dc-43d5-b8bb-8705693cf71b None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:24.553836 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5fd3a4-f2dc-43d5-b8bb-8705693cf71b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:24.562307 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5fd3a4-f2dc-43d5-b8bb-8705693cf71b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:24.563777 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5fd3a4-f2dc-43d5-b8bb-8705693cf71b None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:24.575147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-0c5fd3a4-f2dc-43d5-b8bb-8705693cf71b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 02 08:59:24.575147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:24.575147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:24.575147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:24.575147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:24.575147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:24.575147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:24.575147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:24.575147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:24.575147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:24.575147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:24.575147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:24.575147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:59:24.575147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:59:24.575147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:59:24.575147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:59:24.576745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:24.576745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:24.576745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:59:24.576745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:59:24.576745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:59:24.576745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:59:24.576745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:24.576745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:24.576745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:24.576745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:24.576745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:24.576745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:24.576745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:24.576745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:24.576745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:24.576745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:24.576745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:24.576745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:24.578332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:24.578332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:24.578332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:59:24.578332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:59:24.578332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:59:24.578332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:59:24.578332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:59:24.578332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:59:24.578332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 02 08:59:24.578332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:59:24.578332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 640/1253] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:59:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:24.582076 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d8581-8095-4f3c-b1c4-bad4b8398403 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:24.647947 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:24.694450 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d8581-8095-4f3c-b1c4-bad4b8398403 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:24.733918 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c924f5-914c-4c6a-b23f-70f0ace3c03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:24.734861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c924f5-914c-4c6a-b23f-70f0ace3c03e None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:24.735184 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c924f5-914c-4c6a-b23f-70f0ace3c03e None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:24.735535 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c924f5-914c-4c6a-b23f-70f0ace3c03e None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:24.736341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c924f5-914c-4c6a-b23f-70f0ace3c03e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:24.736650 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c924f5-914c-4c6a-b23f-70f0ace3c03e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:24.738529 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c924f5-914c-4c6a-b23f-70f0ace3c03e None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:24.762921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 614/1254] 10.222.0.36 () {56 vars in 1143 bytes} [Fri Sep 2 08:59:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:24.769195 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5fd3a4-f2dc-43d5-b8bb-8705693cf71b None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:24.889224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5fd3a4-f2dc-43d5-b8bb-8705693cf71b None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:24.925574 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459a865-aeeb-4237-bb93-bb959125af11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:24.926180 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459a865-aeeb-4237-bb93-bb959125af11 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:24.926458 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459a865-aeeb-4237-bb93-bb959125af11 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:24.926730 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459a865-aeeb-4237-bb93-bb959125af11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:24.988427 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:25.000623 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459a865-aeeb-4237-bb93-bb959125af11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:25.002067 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459a865-aeeb-4237-bb93-bb959125af11 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:25.107265 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 641/1255] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:59:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 08:59:25.114101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c924f5-914c-4c6a-b23f-70f0ace3c03e None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:25.219123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c924f5-914c-4c6a-b23f-70f0ace3c03e None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:25.232382 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:25.255448 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921b722-26d9-4fd4-a419-83a46eff3d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:25.256075 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921b722-26d9-4fd4-a419-83a46eff3d78 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:25.256379 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921b722-26d9-4fd4-a419-83a46eff3d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:25.256651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921b722-26d9-4fd4-a419-83a46eff3d78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:25.267360 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921b722-26d9-4fd4-a419-83a46eff3d78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:25.268794 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921b722-26d9-4fd4-a419-83a46eff3d78 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:25.283595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-d921b722-26d9-4fd4-a419-83a46eff3d78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:25.283595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:25.283595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:25.283595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:25.283595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:25.283595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:25.283595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:25.283595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:25.283595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:25.283595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:25.283595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:25.283595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:25.283595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:59:25.283595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:59:25.283595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:59:25.283595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:59:25.284855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:25.284855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:25.284855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:25.284855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:25.284855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:25.284855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:25.284855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:25.284855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:25.284855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:25.284855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:25.284855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:25.284855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:25.284855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:25.284855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:25.284855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:25.284855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:25.284855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:59:25.286192 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:59:25.286192 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:25.286192 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:25.289514 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 615/1256] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:59:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:25.290896 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459a865-aeeb-4237-bb93-bb959125af11 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:25.383450 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:25.395828 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459a865-aeeb-4237-bb93-bb959125af11 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:25.430175 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2bc283-cb0c-44cc-bd8f-3119c11c46b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:25.430884 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bc283-cb0c-44cc-bd8f-3119c11c46b7 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:25.431045 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bc283-cb0c-44cc-bd8f-3119c11c46b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:25.431347 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bc283-cb0c-44cc-bd8f-3119c11c46b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:25.432013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2bc283-cb0c-44cc-bd8f-3119c11c46b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:25.432282 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2bc283-cb0c-44cc-bd8f-3119c11c46b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:25.433641 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2bc283-cb0c-44cc-bd8f-3119c11c46b7 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:25.445970 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 642/1257] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:25.454519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921b722-26d9-4fd4-a419-83a46eff3d78 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:25.557267 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921b722-26d9-4fd4-a419-83a46eff3d78 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:25.592308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa45af-0856-46cf-b80d-3c04ca942f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:25.592931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa45af-0856-46cf-b80d-3c04ca942f2d None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:25.593206 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa45af-0856-46cf-b80d-3c04ca942f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:25.593508 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa45af-0856-46cf-b80d-3c04ca942f2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:25.600063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa45af-0856-46cf-b80d-3c04ca942f2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:25.601450 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa45af-0856-46cf-b80d-3c04ca942f2d None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:25.612395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-b2aa45af-0856-46cf-b80d-3c04ca942f2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:59:25.612395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:25.612395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:25.612395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:25.612395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:25.612395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:25.612395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:25.612395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:25.612395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:25.612395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:25.612395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:25.612395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:25.612395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:59:25.612395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:59:25.612395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:59:25.613468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:59:25.613468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:25.613468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:25.613468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:25.613468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:25.613468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:25.613468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:25.613468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:25.613468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:25.613468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:25.613468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:25.613468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:25.613468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:25.613468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:25.613468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:25.613468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:25.613468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:25.614541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:59:25.614541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:59:25.614541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:59:25.614541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:59:25.614541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:59:25.614541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:59:25.614541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:59:25.614541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:25.614541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 616/1258] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:59:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:59:25.619607 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2bc283-cb0c-44cc-bd8f-3119c11c46b7 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:25.734751 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2bc283-cb0c-44cc-bd8f-3119c11c46b7 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:25.777117 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5ce68-4284-491e-8226-2d63c98cbd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:25.778038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5ce68-4284-491e-8226-2d63c98cbd76 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:25.778431 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5ce68-4284-491e-8226-2d63c98cbd76 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:25.778880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5ce68-4284-491e-8226-2d63c98cbd76 None admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:25.779704 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e5ce68-4284-491e-8226-2d63c98cbd76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:25.780077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e5ce68-4284-491e-8226-2d63c98cbd76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:25.781618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e5ce68-4284-491e-8226-2d63c98cbd76 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:25.796186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 643/1259] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:59:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:25.802718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa45af-0856-46cf-b80d-3c04ca942f2d None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:25.833834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:25.920907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa45af-0856-46cf-b80d-3c04ca942f2d None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:25.956326 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd0b67-1d59-4ce1-b630-2c87c6b76c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:25.956939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd0b67-1d59-4ce1-b630-2c87c6b76c25 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:25.957214 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd0b67-1d59-4ce1-b630-2c87c6b76c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:25.957482 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd0b67-1d59-4ce1-b630-2c87c6b76c25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:25.958221 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd0b67-1d59-4ce1-b630-2c87c6b76c25 None admin] RBAC: Adding query filter params (role.id=a0a76de72fba4036bc1e3e64df717c7f, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=6a0f0fdbbd20492d8c35ec4b73404632) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:25.958527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd0b67-1d59-4ce1-b630-2c87c6b76c25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:25.960021 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbd0b67-1d59-4ce1-b630-2c87c6b76c25 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:25.966131 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 617/1260] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:59:25 2022] GET /identity/v3/role_assignments?user.id=6a0f0fdbbd20492d8c35ec4b73404632&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=a0a76de72fba4036bc1e3e64df717c7f => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:27.829905 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcbde9-d4bf-4863-af97-60624fcee914 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:27.830417 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcbde9-d4bf-4863-af97-60624fcee914 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:27.830417 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcbde9-d4bf-4863-af97-60624fcee914 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:27.831716 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 644/1261] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:59:27.836627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d0473-6c95-4690-b0f4-1185457d40f8 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:27.837065 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d0473-6c95-4690-b0f4-1185457d40f8 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:27.837160 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d0473-6c95-4690-b0f4-1185457d40f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:27.854858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-d81d0473-6c95-4690-b0f4-1185457d40f8 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:59:27.856086 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-d81d0473-6c95-4690-b0f4-1185457d40f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:59:27.919799 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 618/1262] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:59:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:59:28.192159 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcbde9-d4bf-4863-af97-60624fcee914 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:28.202980 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:28.290875 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcbde9-d4bf-4863-af97-60624fcee914 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:28.323831 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72679c1c-42c8-4926-841f-0a5720c24225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:28.324432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72679c1c-42c8-4926-841f-0a5720c24225 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:28.324707 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72679c1c-42c8-4926-841f-0a5720c24225 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:28.324973 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72679c1c-42c8-4926-841f-0a5720c24225 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:28.337278 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72679c1c-42c8-4926-841f-0a5720c24225 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:59:28.342189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-72679c1c-42c8-4926-841f-0a5720c24225 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:59:28.342189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:28.342189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:28.342189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:28.342189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:28.342189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:28.342189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:28.342189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:28.342189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:28.342189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:28.342189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:28.342189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:28.342189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:59:28.342189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:59:28.342189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:59:28.342189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:59:28.343311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:59:28.343311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:59:28.343311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:59:28.343311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:59:28.343311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:28.343311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:28.343311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:28.343311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:28.343311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:28.343311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:28.343311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:28.343311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:28.343311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:28.343311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:28.343311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:28.343311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:28.344500 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:28.344500 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:28.344500 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:28.344500 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:28.344500 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:59:28.344500 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:59:28.344500 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:59:28.344500 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:59:28.344500 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:59:28.344500 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:59:28.344500 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:59:28.344500 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:59:28.344500 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 645/1263] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:59:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:28.349191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d0473-6c95-4690-b0f4-1185457d40f8 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:28.367488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:28.446321 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d0473-6c95-4690-b0f4-1185457d40f8 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:28.478821 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c312cb-7da3-4b86-b380-c054b6953f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:28.479444 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c312cb-7da3-4b86-b380-c054b6953f98 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:28.479726 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c312cb-7da3-4b86-b380-c054b6953f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:28.479997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c312cb-7da3-4b86-b380-c054b6953f98 None admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:28.480660 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c312cb-7da3-4b86-b380-c054b6953f98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:28.480964 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c312cb-7da3-4b86-b380-c054b6953f98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:28.482872 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c312cb-7da3-4b86-b380-c054b6953f98 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:28.490585 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 619/1264] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:59:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:28.496207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72679c1c-42c8-4926-841f-0a5720c24225 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:28.561378 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:28.592910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72679c1c-42c8-4926-841f-0a5720c24225 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:28.625139 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5307185-59f9-4c58-b346-e78074426976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:28.625743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5307185-59f9-4c58-b346-e78074426976 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:28.626017 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5307185-59f9-4c58-b346-e78074426976 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:28.626282 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5307185-59f9-4c58-b346-e78074426976 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:28.635844 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5307185-59f9-4c58-b346-e78074426976 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:28.637262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5307185-59f9-4c58-b346-e78074426976 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:28.651846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-a5307185-59f9-4c58-b346-e78074426976 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:28.651846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:28.651846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:28.651846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:28.651846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:28.651846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:28.651846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:28.651846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:28.651846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:28.651846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:28.651846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:28.651846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:28.651846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:59:28.651846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:59:28.651846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:59:28.651846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:59:28.652936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:28.652936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:28.652936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:28.652936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:28.652936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:28.652936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:28.652936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:28.652936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:28.652936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:28.652936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:28.652936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:28.652936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:28.652936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:28.652936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:28.652936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:28.652936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:28.652936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:59:28.654625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:59:28.654625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:28.654625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:59:28.654625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 646/1265] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:59:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:28.659001 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c312cb-7da3-4b86-b380-c054b6953f98 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:28.740067 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:28.760214 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c312cb-7da3-4b86-b380-c054b6953f98 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:28.796673 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87018445-174d-45ad-ae35-494fe77ad530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:28.797276 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87018445-174d-45ad-ae35-494fe77ad530 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:28.797621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87018445-174d-45ad-ae35-494fe77ad530 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:28.797905 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87018445-174d-45ad-ae35-494fe77ad530 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:28.798627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87018445-174d-45ad-ae35-494fe77ad530 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:28.798997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87018445-174d-45ad-ae35-494fe77ad530 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:28.800585 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87018445-174d-45ad-ae35-494fe77ad530 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:28.812835 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 620/1266] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:28.818595 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5307185-59f9-4c58-b346-e78074426976 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:28.914552 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5307185-59f9-4c58-b346-e78074426976 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:28.933015 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:28.947516 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89dab68-3499-4797-a445-c64bdb0985b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:28.948122 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89dab68-3499-4797-a445-c64bdb0985b4 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:28.948396 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89dab68-3499-4797-a445-c64bdb0985b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:28.948663 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89dab68-3499-4797-a445-c64bdb0985b4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:28.956236 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89dab68-3499-4797-a445-c64bdb0985b4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:28.957647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89dab68-3499-4797-a445-c64bdb0985b4 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:28.968843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-b89dab68-3499-4797-a445-c64bdb0985b4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 02 08:59:28.968843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:28.968843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:28.968843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:28.968843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:28.968843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:28.968843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:28.968843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:28.968843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:28.968843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:28.968843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:28.968843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:28.968843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:59:28.968843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:59:28.968843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:59:28.968843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:59:28.969941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:28.969941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:28.969941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:59:28.969941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:59:28.969941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:59:28.969941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:59:28.969941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:28.969941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:28.969941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:28.969941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:28.969941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:28.969941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:28.969941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:28.969941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:28.969941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:28.969941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:28.969941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:28.969941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:28.970927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:28.970927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:28.970927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:59:28.970927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:59:28.970927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:59:28.970927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:59:28.970927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:59:28.970927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:59:28.970927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 02 08:59:28.970927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:59:28.970927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 647/1267] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:59:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:28.975548 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87018445-174d-45ad-ae35-494fe77ad530 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:29.070236 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87018445-174d-45ad-ae35-494fe77ad530 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:29.103206 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c46b8-fab3-4469-8ab5-6bce498c6959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:29.103801 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c46b8-fab3-4469-8ab5-6bce498c6959 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:29.104082 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c46b8-fab3-4469-8ab5-6bce498c6959 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:29.104356 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c46b8-fab3-4469-8ab5-6bce498c6959 None admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:29.105074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c46b8-fab3-4469-8ab5-6bce498c6959 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:29.105379 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c46b8-fab3-4469-8ab5-6bce498c6959 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:29.106785 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c46b8-fab3-4469-8ab5-6bce498c6959 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:29.115029 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:29.123423 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 621/1268] 10.222.0.36 () {56 vars in 1143 bytes} [Fri Sep 2 08:59:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:59:29.128848 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89dab68-3499-4797-a445-c64bdb0985b4 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:29.225984 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89dab68-3499-4797-a445-c64bdb0985b4 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:29.258115 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8dda3e-8b95-402d-9e27-23e278198f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:29.258713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8dda3e-8b95-402d-9e27-23e278198f85 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:29.258986 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8dda3e-8b95-402d-9e27-23e278198f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:29.259287 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8dda3e-8b95-402d-9e27-23e278198f85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:29.269908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8dda3e-8b95-402d-9e27-23e278198f85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:29.271347 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8dda3e-8b95-402d-9e27-23e278198f85 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:29.284908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-6d8dda3e-8b95-402d-9e27-23e278198f85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:29.284908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:29.284908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:29.284908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:29.284908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:29.284908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:29.284908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:29.284908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:29.284908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:29.284908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:29.284908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:29.284908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:29.284908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:59:29.284908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:59:29.284908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:59:29.284908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:59:29.286005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:29.286005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:29.286005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:29.286005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:29.286005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:29.286005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:29.286005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:29.286005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:29.286005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:29.286005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:29.286005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:29.286005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:29.286005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:29.286005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:29.286005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:29.286005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:29.286005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:59:29.287291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:59:29.287291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:29.287291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:59:29.287291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 648/1269] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:59:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:29.291553 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c46b8-fab3-4469-8ab5-6bce498c6959 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:29.386556 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c46b8-fab3-4469-8ab5-6bce498c6959 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:29.419606 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2cdee6-00d3-4fb2-9ff6-a54bce773236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:29.420194 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2cdee6-00d3-4fb2-9ff6-a54bce773236 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:29.420467 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2cdee6-00d3-4fb2-9ff6-a54bce773236 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:29.420731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2cdee6-00d3-4fb2-9ff6-a54bce773236 None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:29.421356 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2cdee6-00d3-4fb2-9ff6-a54bce773236 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:29.421651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2cdee6-00d3-4fb2-9ff6-a54bce773236 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:29.422989 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2cdee6-00d3-4fb2-9ff6-a54bce773236 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:29.433614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 622/1270] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:29.440115 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8dda3e-8b95-402d-9e27-23e278198f85 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:29.471458 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:29.538093 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8dda3e-8b95-402d-9e27-23e278198f85 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:29.570877 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f9fe9-c334-4a5e-8171-04a725ddf3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:29.571507 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f9fe9-c334-4a5e-8171-04a725ddf3b5 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:29.571787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f9fe9-c334-4a5e-8171-04a725ddf3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:29.572055 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f9fe9-c334-4a5e-8171-04a725ddf3b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:29.578447 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f9fe9-c334-4a5e-8171-04a725ddf3b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:29.579896 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f9fe9-c334-4a5e-8171-04a725ddf3b5 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:29.590790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-dc2f9fe9-c334-4a5e-8171-04a725ddf3b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:59:29.590790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:29.590790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:29.590790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:29.590790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:29.590790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:29.590790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:29.590790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:29.590790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:29.590790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:29.590790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:29.590790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:29.590790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:59:29.590790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:59:29.590790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:59:29.591832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:59:29.591832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:29.591832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:29.591832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:29.591832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:29.591832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:29.591832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:29.591832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:29.591832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:29.591832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:29.591832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:29.591832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:29.591832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:29.591832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:29.591832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:29.591832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:29.591832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:29.592916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:59:29.592916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:59:29.592916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:59:29.592916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:59:29.592916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:59:29.592916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:59:29.592916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:59:29.592916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:59:29.592916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 649/1271] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:59:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 08:59:29.597582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2cdee6-00d3-4fb2-9ff6-a54bce773236 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:29.663301 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:29.704909 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2cdee6-00d3-4fb2-9ff6-a54bce773236 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:29.741792 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4416a5a3-089e-401f-bb45-11c242fa3d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:29.742788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416a5a3-089e-401f-bb45-11c242fa3d9e None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:29.743490 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416a5a3-089e-401f-bb45-11c242fa3d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:29.743938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4416a5a3-089e-401f-bb45-11c242fa3d9e None admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:29.745194 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4416a5a3-089e-401f-bb45-11c242fa3d9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:29.745676 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4416a5a3-089e-401f-bb45-11c242fa3d9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:29.747269 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4416a5a3-089e-401f-bb45-11c242fa3d9e None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:29.763792 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 623/1272] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:59:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:29.770467 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f9fe9-c334-4a5e-8171-04a725ddf3b5 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:29.866805 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:29.876095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f9fe9-c334-4a5e-8171-04a725ddf3b5 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:29.913770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca938e2-e0f5-4f68-9db1-3b1f3cf1dcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:29.914349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca938e2-e0f5-4f68-9db1-3b1f3cf1dcb8 None admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:29.914627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca938e2-e0f5-4f68-9db1-3b1f3cf1dcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:29.915194 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca938e2-e0f5-4f68-9db1-3b1f3cf1dcb8 None admin] PATH_INFO: `/v3/projects/eef80493db09437b88f2b147a59928c1/users/6a0f0fdbbd20492d8c35ec4b73404632/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:29.916774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca938e2-e0f5-4f68-9db1-3b1f3cf1dcb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=eef80493db09437b88f2b147a59928c1, user_id=6a0f0fdbbd20492d8c35ec4b73404632, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:29.918200 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca938e2-e0f5-4f68-9db1-3b1f3cf1dcb8 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:29.931214 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 650/1273] 10.222.0.36 () {58 vars in 1439 bytes} [Fri Sep 2 08:59:29 2022] PUT /identity/v3/projects/eef80493db09437b88f2b147a59928c1/users/6a0f0fdbbd20492d8c35ec4b73404632/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 08:59:31.786958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ede74-f45e-4461-8a80-5f26fa9a63ef None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:31.787500 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ede74-f45e-4461-8a80-5f26fa9a63ef None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:31.787500 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ede74-f45e-4461-8a80-5f26fa9a63ef None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:31.788958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 624/1274] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:59:31.795586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b91dc-fbba-48bb-a5bd-3d4feb9bce5d None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:31.796025 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b91dc-fbba-48bb-a5bd-3d4feb9bce5d None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:31.796191 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b91dc-fbba-48bb-a5bd-3d4feb9bce5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:31.816522 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-716b91dc-fbba-48bb-a5bd-3d4feb9bce5d None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:59:31.817711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-716b91dc-fbba-48bb-a5bd-3d4feb9bce5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:59:31.882296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 651/1275] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:59:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:59:32.146400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ede74-f45e-4461-8a80-5f26fa9a63ef None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:32.243385 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ede74-f45e-4461-8a80-5f26fa9a63ef None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:32.260066 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:32.277215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d9706-4ee6-4655-8332-9880dcb8b92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:32.277835 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d9706-4ee6-4655-8332-9880dcb8b92b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:32.278112 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d9706-4ee6-4655-8332-9880dcb8b92b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:32.278382 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d9706-4ee6-4655-8332-9880dcb8b92b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:32.288134 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d9706-4ee6-4655-8332-9880dcb8b92b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 08:59:32.292935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-ae8d9706-4ee6-4655-8332-9880dcb8b92b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:59:32.292935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:32.292935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:32.292935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:32.292935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:32.292935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:32.292935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:32.292935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:32.292935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:32.292935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:32.292935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:32.292935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:32.292935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 08:59:32.292935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 08:59:32.292935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 08:59:32.292935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 08:59:32.293997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 08:59:32.293997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 08:59:32.293997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 08:59:32.293997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 08:59:32.293997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:32.293997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:32.293997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:32.293997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:32.293997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:32.293997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:32.293997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:32.293997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:32.293997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:32.293997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:32.293997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:32.293997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:32.295009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:32.295009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:32.295009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:32.295009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:32.295009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 08:59:32.295009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 08:59:32.295009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 08:59:32.295009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 08:59:32.295009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 08:59:32.295009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 08:59:32.295009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 08:59:32.295009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:32.295009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 625/1276] 10.222.0.36 () {56 vars in 1096 bytes} [Fri Sep 2 08:59:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:32.299911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b91dc-fbba-48bb-a5bd-3d4feb9bce5d None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:32.378934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:32.397915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b91dc-fbba-48bb-a5bd-3d4feb9bce5d None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:32.430116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42160bf-c464-4d91-ad83-fa3ce8ac1451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:32.430712 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42160bf-c464-4d91-ad83-fa3ce8ac1451 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:32.430994 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42160bf-c464-4d91-ad83-fa3ce8ac1451 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:32.431293 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42160bf-c464-4d91-ad83-fa3ce8ac1451 None admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:32.431954 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42160bf-c464-4d91-ad83-fa3ce8ac1451 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:32.432257 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42160bf-c464-4d91-ad83-fa3ce8ac1451 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:32.433623 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42160bf-c464-4d91-ad83-fa3ce8ac1451 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:32.442139 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 652/1277] 10.222.0.36 () {56 vars in 1099 bytes} [Fri Sep 2 08:59:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:32.447801 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d9706-4ee6-4655-8332-9880dcb8b92b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:32.542917 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d9706-4ee6-4655-8332-9880dcb8b92b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:32.575682 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b22865-d5c4-4463-acc7-e7048057697a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:32.576287 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b22865-d5c4-4463-acc7-e7048057697a None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:32.576564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b22865-d5c4-4463-acc7-e7048057697a None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:32.576833 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b22865-d5c4-4463-acc7-e7048057697a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:32.586301 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b22865-d5c4-4463-acc7-e7048057697a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:32.587730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b22865-d5c4-4463-acc7-e7048057697a None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:32.601588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-97b22865-d5c4-4463-acc7-e7048057697a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:32.601588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:32.601588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:32.601588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:32.601588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:32.601588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:32.601588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:32.601588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:32.601588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:32.601588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:32.601588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:32.601588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:32.601588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:59:32.601588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:59:32.601588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:59:32.601588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:59:32.602697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:32.602697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:32.602697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:32.602697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:32.602697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:32.602697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:32.602697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:32.602697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:32.602697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:32.602697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:32.602697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:32.602697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:32.602697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:32.602697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:32.602697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:32.602697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:32.602697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:59:32.603836 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:59:32.603836 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:32.603836 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:32.603836 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 626/1278] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:59:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:32.608934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42160bf-c464-4d91-ad83-fa3ce8ac1451 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:32.704682 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42160bf-c464-4d91-ad83-fa3ce8ac1451 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:32.739297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0ad49-681f-474b-af7c-e648b81a0062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:32.739881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0ad49-681f-474b-af7c-e648b81a0062 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:32.740172 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0ad49-681f-474b-af7c-e648b81a0062 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:32.740398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0ad49-681f-474b-af7c-e648b81a0062 None admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:32.741035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0ad49-681f-474b-af7c-e648b81a0062 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:32.741332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0ad49-681f-474b-af7c-e648b81a0062 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:32.742848 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0ad49-681f-474b-af7c-e648b81a0062 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:32.755608 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 653/1279] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:32.761512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b22865-d5c4-4463-acc7-e7048057697a None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:32.793081 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:32.860338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b22865-d5c4-4463-acc7-e7048057697a None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:32.892147 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e65107-2a96-4b98-b727-944e88a82026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:32.892766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e65107-2a96-4b98-b727-944e88a82026 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:32.893041 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e65107-2a96-4b98-b727-944e88a82026 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:32.893309 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e65107-2a96-4b98-b727-944e88a82026 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:32.901105 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e65107-2a96-4b98-b727-944e88a82026 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:32.902520 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e65107-2a96-4b98-b727-944e88a82026 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:32.913200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-70e65107-2a96-4b98-b727-944e88a82026 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 02 08:59:32.913200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:32.913200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:32.913200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:32.913200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:32.913200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:32.913200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:32.913200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:32.913200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:32.913200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:32.913200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:32.913200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:32.913200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 08:59:32.913200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 08:59:32.913200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 08:59:32.913200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 08:59:32.914295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:32.914295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:32.914295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 08:59:32.914295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:59:32.914295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 08:59:32.914295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 08:59:32.914295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:32.914295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:32.914295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:32.914295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:32.914295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:32.914295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:32.914295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:32.914295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:32.914295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:32.914295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:32.914295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:32.914295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:32.915315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:32.915315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:32.915315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 08:59:32.915315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 08:59:32.915315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 08:59:32.915315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 08:59:32.915315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 08:59:32.915315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 08:59:32.915315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 02 08:59:32.915315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:32.915315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 627/1280] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:59:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:32.920535 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0ad49-681f-474b-af7c-e648b81a0062 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:32.924314 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:33.017391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0ad49-681f-474b-af7c-e648b81a0062 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:33.050725 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691195ed-750d-48af-8c48-252e5173b8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:33.051335 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691195ed-750d-48af-8c48-252e5173b8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:33.051613 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691195ed-750d-48af-8c48-252e5173b8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:33.051882 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691195ed-750d-48af-8c48-252e5173b8c5 None admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:33.052592 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691195ed-750d-48af-8c48-252e5173b8c5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:33.052892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691195ed-750d-48af-8c48-252e5173b8c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:33.054310 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691195ed-750d-48af-8c48-252e5173b8c5 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:33.070964 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 654/1281] 10.222.0.36 () {56 vars in 1143 bytes} [Fri Sep 2 08:59:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:33.076861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e65107-2a96-4b98-b727-944e88a82026 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:33.163256 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:33.174109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e65107-2a96-4b98-b727-944e88a82026 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:33.206151 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c1a0c-0354-4e3c-b283-f4ddfda3532b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:33.206738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c1a0c-0354-4e3c-b283-f4ddfda3532b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:33.207011 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c1a0c-0354-4e3c-b283-f4ddfda3532b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:33.207305 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c1a0c-0354-4e3c-b283-f4ddfda3532b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:33.270865 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59c1a0c-0354-4e3c-b283-f4ddfda3532b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:33.272282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59c1a0c-0354-4e3c-b283-f4ddfda3532b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:33.356778 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:33.368406 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 628/1282] 10.222.0.36 () {58 vars in 1282 bytes} [Fri Sep 2 08:59:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 08:59:33.374172 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691195ed-750d-48af-8c48-252e5173b8c5 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:33.434935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:33.473183 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691195ed-750d-48af-8c48-252e5173b8c5 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:33.505422 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d82856-f30c-42ea-939d-9412ae3cb747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:33.506041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d82856-f30c-42ea-939d-9412ae3cb747 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:33.506320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d82856-f30c-42ea-939d-9412ae3cb747 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:33.506595 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d82856-f30c-42ea-939d-9412ae3cb747 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:33.516752 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d82856-f30c-42ea-939d-9412ae3cb747 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:33.518184 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d82856-f30c-42ea-939d-9412ae3cb747 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:33.532942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-88d82856-f30c-42ea-939d-9412ae3cb747 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:33.532942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:33.532942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:33.532942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:33.532942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:33.532942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:33.532942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:33.532942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:33.532942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:33.532942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:33.532942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:33.532942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:33.532942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 08:59:33.532942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 08:59:33.532942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 08:59:33.532942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 08:59:33.534070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:33.534070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:33.534070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:33.534070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:33.534070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:33.534070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:33.534070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:33.534070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:33.534070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:33.534070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:33.534070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:33.534070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:33.534070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:33.534070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:33.534070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:33.534070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:33.534070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 08:59:33.535205 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 08:59:33.535205 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 08:59:33.535205 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:59:33.535205 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 655/1283] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 08:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:33.539901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c1a0c-0354-4e3c-b283-f4ddfda3532b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:33.634722 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c1a0c-0354-4e3c-b283-f4ddfda3532b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:33.667221 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4b0a0-2ac3-4a40-9f70-6cd4b926542b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:33.667807 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4b0a0-2ac3-4a40-9f70-6cd4b926542b None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:33.668080 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4b0a0-2ac3-4a40-9f70-6cd4b926542b None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:33.668344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce4b0a0-2ac3-4a40-9f70-6cd4b926542b None admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:33.668971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce4b0a0-2ac3-4a40-9f70-6cd4b926542b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:33.669269 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce4b0a0-2ac3-4a40-9f70-6cd4b926542b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:33.670618 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce4b0a0-2ac3-4a40-9f70-6cd4b926542b None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:33.681231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 629/1284] 10.222.0.36 () {56 vars in 1109 bytes} [Fri Sep 2 08:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:33.687460 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d82856-f30c-42ea-939d-9412ae3cb747 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:33.790573 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d82856-f30c-42ea-939d-9412ae3cb747 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:33.803744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:33.824006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaafda6-a76a-4c7f-b4fc-11fd32c5d714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:33.824626 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaafda6-a76a-4c7f-b4fc-11fd32c5d714 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:33.824905 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaafda6-a76a-4c7f-b4fc-11fd32c5d714 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:33.825176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaafda6-a76a-4c7f-b4fc-11fd32c5d714 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:33.832276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaafda6-a76a-4c7f-b4fc-11fd32c5d714 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:33.833703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaafda6-a76a-4c7f-b4fc-11fd32c5d714 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:33.844693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application [None req-beaafda6-a76a-4c7f-b4fc-11fd32c5d714 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:59:33.844693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:33.844693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:33.844693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:33.844693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:33.844693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:33.844693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:33.844693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:33.844693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:33.844693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:33.844693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:33.844693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:33.844693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 08:59:33.844693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 08:59:33.844693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 08:59:33.845731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 08:59:33.845731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:33.845731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:33.845731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:33.845731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:33.845731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:33.845731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:33.845731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:33.845731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:33.845731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:33.845731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:33.845731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:33.845731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:33.845731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:33.845731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:33.845731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:33.845731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:33.846694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 08:59:33.846694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 08:59:33.846694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 08:59:33.846694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 08:59:33.846694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 08:59:33.846694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 08:59:33.846694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 08:59:33.846694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: ERROR keystone.server.flask.application Sep 02 08:59:33.846694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 656/1285] 10.222.0.36 () {56 vars in 1111 bytes} [Fri Sep 2 08:59:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:33.851625 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4b0a0-2ac3-4a40-9f70-6cd4b926542b None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:33.900022 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:33.949738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce4b0a0-2ac3-4a40-9f70-6cd4b926542b None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:33.982107 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879090d1-5ca6-4ffd-9d07-b111648dd7f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:33.982705 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879090d1-5ca6-4ffd-9d07-b111648dd7f0 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:33.982986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879090d1-5ca6-4ffd-9d07-b111648dd7f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:33.983286 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879090d1-5ca6-4ffd-9d07-b111648dd7f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:33.983977 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879090d1-5ca6-4ffd-9d07-b111648dd7f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:33.984284 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879090d1-5ca6-4ffd-9d07-b111648dd7f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:33.985704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879090d1-5ca6-4ffd-9d07-b111648dd7f0 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:33.997175 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 630/1286] 10.222.0.36 () {56 vars in 1148 bytes} [Fri Sep 2 08:59:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 08:59:34.003647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaafda6-a76a-4c7f-b4fc-11fd32c5d714 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:34.100172 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaafda6-a76a-4c7f-b4fc-11fd32c5d714 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:34.132357 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf81bac-eec4-43a9-b90b-dfe568d054a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:34.132936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf81bac-eec4-43a9-b90b-dfe568d054a0 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:34.133212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf81bac-eec4-43a9-b90b-dfe568d054a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:34.133476 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf81bac-eec4-43a9-b90b-dfe568d054a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:34.134176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf81bac-eec4-43a9-b90b-dfe568d054a0 None admin] RBAC: Adding query filter params (role.id=a0a76de72fba4036bc1e3e64df717c7f, scope.project.id=eef80493db09437b88f2b147a59928c1, user.id=6a0f0fdbbd20492d8c35ec4b73404632) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:34.134476 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf81bac-eec4-43a9-b90b-dfe568d054a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:34.135902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf81bac-eec4-43a9-b90b-dfe568d054a0 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:34.141386 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 657/1287] 10.222.0.36 () {56 vars in 1374 bytes} [Fri Sep 2 08:59:34 2022] GET /identity/v3/role_assignments?user.id=6a0f0fdbbd20492d8c35ec4b73404632&scope.project.id=eef80493db09437b88f2b147a59928c1&role.id=a0a76de72fba4036bc1e3e64df717c7f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:36.032161 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b556e9f-33ee-4f65-84f7-07291a9c478d None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:36.032766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b556e9f-33ee-4f65-84f7-07291a9c478d None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:36.032766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b556e9f-33ee-4f65-84f7-07291a9c478d None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:36.034021 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 631/1288] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:59:36.039130 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff9c26-f12d-4a5a-afb2-340afaca6da7 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:36.039530 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff9c26-f12d-4a5a-afb2-340afaca6da7 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:36.039680 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff9c26-f12d-4a5a-afb2-340afaca6da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:36.057743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-03ff9c26-f12d-4a5a-afb2-340afaca6da7 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:59:36.058905 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-03ff9c26-f12d-4a5a-afb2-340afaca6da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:59:36.087029 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:36.123415 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 658/1289] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:59:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:59:36.390425 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b556e9f-33ee-4f65-84f7-07291a9c478d None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:36.486528 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b556e9f-33ee-4f65-84f7-07291a9c478d None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:36.497337 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:36.519923 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe3840-7679-4078-8d8a-dce2c5c74177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:36.520531 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe3840-7679-4078-8d8a-dce2c5c74177 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:36.520805 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe3840-7679-4078-8d8a-dce2c5c74177 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:36.521071 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe3840-7679-4078-8d8a-dce2c5c74177 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:36.521640 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe3840-7679-4078-8d8a-dce2c5c74177 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:36.522977 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe3840-7679-4078-8d8a-dce2c5c74177 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:36.531961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-eabe3840-7679-4078-8d8a-dce2c5c74177 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 08:59:36.531961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:36.531961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:59:36.531961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:59:36.531961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:59:36.531961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:59:36.531961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:59:36.531961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:59:36.531961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 08:59:36.531961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:36.531961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:59:36.531961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:36.531961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:36.531961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:36.531961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:36.531961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:36.531961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:36.533231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:36.533231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:36.533231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:36.533231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:36.533231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:36.533231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:36.533231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:59:36.533231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:59:36.533231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:59:36.533231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:59:36.533231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:36.533231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:36.533231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:36.533231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:36.533231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:36.533231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:36.534384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:36.534384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:36.534384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:36.534384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:36.534384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:36.534384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:36.534384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:36.534384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:36.534384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:36.534384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:36.534384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:59:36.534384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:59:36.534384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 08:59:36.534384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:36.534384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 632/1290] 10.222.0.36 () {56 vars in 1117 bytes} [Fri Sep 2 08:59:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:36.538923 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff9c26-f12d-4a5a-afb2-340afaca6da7 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:36.632383 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff9c26-f12d-4a5a-afb2-340afaca6da7 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:36.664758 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74187165-140f-4928-8703-2fbb50d78c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:36.665342 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74187165-140f-4928-8703-2fbb50d78c37 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:36.665616 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74187165-140f-4928-8703-2fbb50d78c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:36.665884 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74187165-140f-4928-8703-2fbb50d78c37 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:36.666492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74187165-140f-4928-8703-2fbb50d78c37 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:36.666785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74187165-140f-4928-8703-2fbb50d78c37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:36.668163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74187165-140f-4928-8703-2fbb50d78c37 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:36.673230 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 659/1291] 10.222.0.36 () {56 vars in 1116 bytes} [Fri Sep 2 08:59:36 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:36.678730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe3840-7679-4078-8d8a-dce2c5c74177 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:36.780392 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe3840-7679-4078-8d8a-dce2c5c74177 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:36.814963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9feda-8b22-4c26-af8c-bddcf137a44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:36.815655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9feda-8b22-4c26-af8c-bddcf137a44c None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:36.815930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9feda-8b22-4c26-af8c-bddcf137a44c None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:36.816197 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9feda-8b22-4c26-af8c-bddcf137a44c None admin] PATH_INFO: `/v3/services` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:36.816815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9feda-8b22-4c26-af8c-bddcf137a44c None admin] RBAC: Adding query filter params (type=placement) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:36.817111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9feda-8b22-4c26-af8c-bddcf137a44c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:36.818462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9feda-8b22-4c26-af8c-bddcf137a44c None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:36.822690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 633/1292] 10.222.0.36 () {56 vars in 1116 bytes} [Fri Sep 2 08:59:36 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:38.695406 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef835a6-9157-476b-97c5-d02ad09d0786 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:38.695928 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef835a6-9157-476b-97c5-d02ad09d0786 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:38.695928 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef835a6-9157-476b-97c5-d02ad09d0786 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:38.697202 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 660/1293] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:59:38.702200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1991f19-f6ce-4ce8-a86a-4cae8c7ecf33 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:38.702586 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1991f19-f6ce-4ce8-a86a-4cae8c7ecf33 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:38.702747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1991f19-f6ce-4ce8-a86a-4cae8c7ecf33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:38.720347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-c1991f19-f6ce-4ce8-a86a-4cae8c7ecf33 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:59:38.721502 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c1991f19-f6ce-4ce8-a86a-4cae8c7ecf33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:59:38.752600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:38.790240 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 634/1294] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:59:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:59:39.063700 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef835a6-9157-476b-97c5-d02ad09d0786 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:39.094225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:39.165111 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef835a6-9157-476b-97c5-d02ad09d0786 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:39.200128 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b638d698-01c8-45ae-a5b2-f453d9a8e150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:39.200767 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638d698-01c8-45ae-a5b2-f453d9a8e150 None admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:39.201045 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638d698-01c8-45ae-a5b2-f453d9a8e150 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:39.201314 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b638d698-01c8-45ae-a5b2-f453d9a8e150 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:39.201966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638d698-01c8-45ae-a5b2-f453d9a8e150 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:39.203349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b638d698-01c8-45ae-a5b2-f453d9a8e150 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:39.217351 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 661/1295] 10.222.0.36 () {60 vars in 1141 bytes} [Fri Sep 2 08:59:39 2022] POST /identity/v3/services => generated 246 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:59:41.103709 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ba48e-51dd-4a25-aa97-f1fb44635859 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:41.104243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ba48e-51dd-4a25-aa97-f1fb44635859 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:41.104243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ba48e-51dd-4a25-aa97-f1fb44635859 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:41.105599 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 635/1296] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:59:41.110709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293a3e1-a60d-4e40-8db3-19bdd41d8443 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:41.111151 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293a3e1-a60d-4e40-8db3-19bdd41d8443 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:41.111285 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293a3e1-a60d-4e40-8db3-19bdd41d8443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:41.128855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-7293a3e1-a60d-4e40-8db3-19bdd41d8443 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:59:41.130017 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-7293a3e1-a60d-4e40-8db3-19bdd41d8443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:59:41.206372 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 662/1297] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:59:41 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:59:41.476833 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ba48e-51dd-4a25-aa97-f1fb44635859 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:41.574936 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ba48e-51dd-4a25-aa97-f1fb44635859 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:41.607278 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff1546-ec98-488a-848b-33f190654060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:41.607844 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff1546-ec98-488a-848b-33f190654060 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:41.608118 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff1546-ec98-488a-848b-33f190654060 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:41.608387 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff1546-ec98-488a-848b-33f190654060 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:41.608961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff1546-ec98-488a-848b-33f190654060 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:41.610316 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff1546-ec98-488a-848b-33f190654060 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:41.618864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-67ff1546-ec98-488a-848b-33f190654060 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 08:59:41.618864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:41.618864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:59:41.618864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:59:41.618864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:59:41.618864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:59:41.618864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:59:41.618864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:59:41.618864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 08:59:41.618864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:41.618864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:59:41.618864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:41.618864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:41.618864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:41.618864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:41.618864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:41.618864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:41.620047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:41.620047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:41.620047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:41.620047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:41.620047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:41.620047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:41.620047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:59:41.620047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:59:41.620047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:59:41.620047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:59:41.620047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:41.620047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:41.620047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:41.620047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:41.620047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:41.620047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:41.620912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:41.620912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:41.620912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:41.620912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:41.620912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:41.620912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:41.620912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:41.620912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:41.620912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:41.620912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:41.620912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:59:41.620912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:59:41.620912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 08:59:41.620912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:41.620912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 636/1298] 10.222.0.36 () {56 vars in 1117 bytes} [Fri Sep 2 08:59:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:41.626451 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293a3e1-a60d-4e40-8db3-19bdd41d8443 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:41.644103 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:41.721591 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293a3e1-a60d-4e40-8db3-19bdd41d8443 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:41.756519 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9ff96-3562-4d00-817b-47085a67ede3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:41.757020 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9ff96-3562-4d00-817b-47085a67ede3 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:41.757295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9ff96-3562-4d00-817b-47085a67ede3 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:41.757569 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f9ff96-3562-4d00-817b-47085a67ede3 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:41.758186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9ff96-3562-4d00-817b-47085a67ede3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:41.758481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9ff96-3562-4d00-817b-47085a67ede3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:41.759910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f9ff96-3562-4d00-817b-47085a67ede3 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:41.765016 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 663/1299] 10.222.0.36 () {56 vars in 1116 bytes} [Fri Sep 2 08:59:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:41.771204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff1546-ec98-488a-848b-33f190654060 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:41.803035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:41.873437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff1546-ec98-488a-848b-33f190654060 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:41.906128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0418843-fe68-459f-80e9-66e1ce701f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:41.906719 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0418843-fe68-459f-80e9-66e1ce701f60 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:41.906998 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0418843-fe68-459f-80e9-66e1ce701f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:41.907294 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0418843-fe68-459f-80e9-66e1ce701f60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:41.907987 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0418843-fe68-459f-80e9-66e1ce701f60 None admin] RBAC: Adding query filter params (interface=public, service_id=421703c4b8ae46b6889209d569886e39, region_id=RegionOne) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:41.908294 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0418843-fe68-459f-80e9-66e1ce701f60 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:41.909640 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0418843-fe68-459f-80e9-66e1ce701f60 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:41.914461 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 637/1300] 10.222.0.36 () {56 vars in 1251 bytes} [Fri Sep 2 08:59:41 2022] GET /identity/v3/endpoints?service_id=421703c4b8ae46b6889209d569886e39&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:41.920206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9ff96-3562-4d00-817b-47085a67ede3 None admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:41.999845 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:42.017994 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f9ff96-3562-4d00-817b-47085a67ede3 None admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:42.049783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979678aa-3c84-4ddd-8a52-bcc6efa14ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:42.050367 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979678aa-3c84-4ddd-8a52-bcc6efa14ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:42.050641 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979678aa-3c84-4ddd-8a52-bcc6efa14ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:42.050908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979678aa-3c84-4ddd-8a52-bcc6efa14ec6 None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:42.051527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979678aa-3c84-4ddd-8a52-bcc6efa14ec6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:42.052870 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979678aa-3c84-4ddd-8a52-bcc6efa14ec6 None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:42.059647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 664/1301] 10.222.0.36 () {56 vars in 1087 bytes} [Fri Sep 2 08:59:41 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 08:59:43.924194 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11bc98-dfda-4baa-9198-702432119b34 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:43.924746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11bc98-dfda-4baa-9198-702432119b34 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:43.924746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11bc98-dfda-4baa-9198-702432119b34 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:43.926049 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 638/1302] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 08:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 08:59:43.931228 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0e76f-c81c-420c-9836-1ea275093976 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:43.931628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0e76f-c81c-420c-9836-1ea275093976 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:43.931788 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c0e76f-c81c-420c-9836-1ea275093976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:43.950343 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-c1c0e76f-c81c-420c-9836-1ea275093976 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 08:59:43.951781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c1c0e76f-c81c-420c-9836-1ea275093976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 08:59:44.016488 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 665/1303] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 08:59:43 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 08:59:44.283794 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11bc98-dfda-4baa-9198-702432119b34 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:44.376365 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:44.383250 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11bc98-dfda-4baa-9198-702432119b34 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:44.415243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66dc6e6-0318-4ca3-890b-1afd17482ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:44.415858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dc6e6-0318-4ca3-890b-1afd17482ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:44.416133 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dc6e6-0318-4ca3-890b-1afd17482ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:44.416400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dc6e6-0318-4ca3-890b-1afd17482ea9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:44.416971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66dc6e6-0318-4ca3-890b-1afd17482ea9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:44.418316 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66dc6e6-0318-4ca3-890b-1afd17482ea9 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:44.427462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application [None req-f66dc6e6-0318-4ca3-890b-1afd17482ea9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 08:59:44.427462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:44.427462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 08:59:44.427462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 08:59:44.427462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 08:59:44.427462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 08:59:44.427462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 08:59:44.427462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:59:44.427462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 08:59:44.427462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:44.427462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 08:59:44.427462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:44.427462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 08:59:44.427462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 08:59:44.427462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 08:59:44.427462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 08:59:44.427462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 08:59:44.428653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 08:59:44.428653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 08:59:44.428653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 08:59:44.428653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 08:59:44.428653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 08:59:44.428653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 08:59:44.428653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 08:59:44.428653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 08:59:44.428653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 08:59:44.428653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 08:59:44.428653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 08:59:44.428653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 08:59:44.428653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 08:59:44.428653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 08:59:44.428653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 08:59:44.428653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 08:59:44.429672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 08:59:44.429672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application with Lock( Sep 02 08:59:44.429672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 08:59:44.429672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self._enter() Sep 02 08:59:44.429672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 08:59:44.429672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 08:59:44.429672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 08:59:44.429672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application return self.creator() Sep 02 08:59:44.429672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 08:59:44.429672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application created_value = creator( Sep 02 08:59:44.429672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 08:59:44.429672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 08:59:44.429672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 08:59:44.429672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: ERROR keystone.server.flask.application Sep 02 08:59:44.429672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 639/1304] 10.222.0.36 () {56 vars in 1117 bytes} [Fri Sep 2 08:59:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 08:59:44.434715 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c0e76f-c81c-420c-9836-1ea275093976 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:44.508289 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:44.530774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c0e76f-c81c-420c-9836-1ea275093976 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:44.563709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912700c-06b5-474c-98da-f22ff2e3300d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:44.564250 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912700c-06b5-474c-98da-f22ff2e3300d None admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:44.564527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912700c-06b5-474c-98da-f22ff2e3300d None admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:44.564802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912700c-06b5-474c-98da-f22ff2e3300d None admin] PATH_INFO: `/v3/services` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:44.565420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912700c-06b5-474c-98da-f22ff2e3300d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 08:59:44.565721 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912700c-06b5-474c-98da-f22ff2e3300d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:44.567099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912700c-06b5-474c-98da-f22ff2e3300d None admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:44.572028 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 666/1305] 10.222.0.36 () {56 vars in 1116 bytes} [Fri Sep 2 08:59:44 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 08:59:44.578146 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66dc6e6-0318-4ca3-890b-1afd17482ea9 None admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 08:59:44.674332 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66dc6e6-0318-4ca3-890b-1afd17482ea9 None admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 08:59:44.707168 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e7020-db05-4efc-9ab6-41c86b2f3041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 08:59:44.707757 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e7020-db05-4efc-9ab6-41c86b2f3041 None admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 08:59:44.708034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e7020-db05-4efc-9ab6-41c86b2f3041 None admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 08:59:44.708304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e7020-db05-4efc-9ab6-41c86b2f3041 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 08:59:44.708965 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e7020-db05-4efc-9ab6-41c86b2f3041 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 08:59:44.710304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e7020-db05-4efc-9ab6-41c86b2f3041 None admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 08:59:44.730899 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 640/1306] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 08:59:44 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 08:59:54.569220 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=30 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 08:59:54.726629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:00:48.113947 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94992d28-d779-47b0-a729-241f8d3c965f None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:00:48.114425 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94992d28-d779-47b0-a729-241f8d3c965f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:00:48.114506 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94992d28-d779-47b0-a729-241f8d3c965f None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:00:48.120902 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e88fdd-6224-4bee-94d0-ca3f71f73a89 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:00:48.121307 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e88fdd-6224-4bee-94d0-ca3f71f73a89 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:00:48.121453 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e88fdd-6224-4bee-94d0-ca3f71f73a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:00:48.126737 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 667/1307] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:00:48 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:00:48.213594 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-40e88fdd-6224-4bee-94d0-ca3f71f73a89 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:00:48.214815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-40e88fdd-6224-4bee-94d0-ca3f71f73a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:00:48.351589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 641/1308] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:00:48.636549 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df49a3-0399-48b8-a5a6-ef3e22309fb8 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:00:48.636908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df49a3-0399-48b8-a5a6-ef3e22309fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:00:48.637099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df49a3-0399-48b8-a5a6-ef3e22309fb8 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:00:48.640133 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 668/1309] 10.222.0.36 () {54 vars in 965 bytes} [Fri Sep 2 09:00:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:00:48.646365 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d1024-525d-4930-9dca-4bd45d1a9bf1 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:00:48.646934 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d1024-525d-4930-9dca-4bd45d1a9bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:00:48.647441 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d1024-525d-4930-9dca-4bd45d1a9bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:00:48.727023 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-b57d1024-525d-4930-9dca-4bd45d1a9bf1 None None] MFA Rules not processed for user `07a825fddbc04f918bcd631976038cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:00:48.728261 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b57d1024-525d-4930-9dca-4bd45d1a9bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:00:48.750695 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:00:48.823099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df49a3-0399-48b8-a5a6-ef3e22309fb8 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:00:48.825739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a8df49a3-0399-48b8-a5a6-ef3e22309fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:00:48.833973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 642/1310] 10.222.0.36 () {58 vars in 1062 bytes} [Fri Sep 2 09:00:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:00:48.959187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df49a3-0399-48b8-a5a6-ef3e22309fb8 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:00:49.086092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4128b-34f9-4f07-ba38-235600d4e129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:00:49.086835 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4128b-34f9-4f07-ba38-235600d4e129 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:00:49.087256 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4128b-34f9-4f07-ba38-235600d4e129 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:00:49.087622 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4128b-34f9-4f07-ba38-235600d4e129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:00:49.089306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-bdd4128b-34f9-4f07-ba38-235600d4e129 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:00:49.169528 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:00:49.238118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4128b-34f9-4f07-ba38-235600d4e129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:00:49.239697 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4128b-34f9-4f07-ba38-235600d4e129 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:00:49.262863 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 669/1311] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:00:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:00:51.507610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbb9fb-c7fc-4e14-b399-31967da9d5dd None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:00:51.508200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbb9fb-c7fc-4e14-b399-31967da9d5dd None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:00:51.508292 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fbb9fb-c7fc-4e14-b399-31967da9d5dd None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:00:51.509607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 643/1312] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:00:51.514869 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7893079-bc83-4cf1-8789-c6e4edc75900 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:00:51.515330 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7893079-bc83-4cf1-8789-c6e4edc75900 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:00:51.515467 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7893079-bc83-4cf1-8789-c6e4edc75900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:00:51.591434 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-f7893079-bc83-4cf1-8789-c6e4edc75900 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:00:51.592676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f7893079-bc83-4cf1-8789-c6e4edc75900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:00:51.696019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 670/1313] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:00:51.978460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbb9fb-c7fc-4e14-b399-31967da9d5dd None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:00:51.980036 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b9fbb9fb-c7fc-4e14-b399-31967da9d5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:00:52.085143 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fbb9fb-c7fc-4e14-b399-31967da9d5dd None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:00:52.154199 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:00:52.157128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb9d78-4a7e-4ff6-b2cc-9f5d5b08c354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:00:52.157943 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb9d78-4a7e-4ff6-b2cc-9f5d5b08c354 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:00:52.158274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb9d78-4a7e-4ff6-b2cc-9f5d5b08c354 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:00:52.158695 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb9d78-4a7e-4ff6-b2cc-9f5d5b08c354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:00:52.160524 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-decb9d78-4a7e-4ff6-b2cc-9f5d5b08c354 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:00:52.271048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decb9d78-4a7e-4ff6-b2cc-9f5d5b08c354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:00:52.272421 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decb9d78-4a7e-4ff6-b2cc-9f5d5b08c354 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:00:52.290123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 644/1314] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:00:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:00:54.488717 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7fa44-d590-4e91-bbaf-1725baac2a20 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:00:54.489257 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7fa44-d590-4e91-bbaf-1725baac2a20 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:00:54.489257 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7fa44-d590-4e91-bbaf-1725baac2a20 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:00:54.490600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 671/1315] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:00:54.495738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e338c41-6981-4186-b556-03d85ade921d None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:00:54.496145 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e338c41-6981-4186-b556-03d85ade921d None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:00:54.496315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e338c41-6981-4186-b556-03d85ade921d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:00:54.561395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-9e338c41-6981-4186-b556-03d85ade921d None None] MFA Rules not processed for user `9b90e57a93b44d24bc368ab2e96b81d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:00:54.562637 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-9e338c41-6981-4186-b556-03d85ade921d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:00:54.612634 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:00:54.656893 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 645/1316] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:00:54 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:00:54.941538 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7fa44-d590-4e91-bbaf-1725baac2a20 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:00:54.943046 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0fd7fa44-d590-4e91-bbaf-1725baac2a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:00:54.952811 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:00:55.043938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7fa44-d590-4e91-bbaf-1725baac2a20 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:00:55.115294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc82aef-c3b5-4ed2-be34-b65f02be35c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:00:55.115889 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc82aef-c3b5-4ed2-be34-b65f02be35c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:00:55.116166 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc82aef-c3b5-4ed2-be34-b65f02be35c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:00:55.116432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc82aef-c3b5-4ed2-be34-b65f02be35c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:00:55.117978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5dc82aef-c3b5-4ed2-be34-b65f02be35c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:00:55.181726 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:00:55.232303 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc82aef-c3b5-4ed2-be34-b65f02be35c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:00:55.233754 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc82aef-c3b5-4ed2-be34-b65f02be35c5 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:00:55.254541 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 672/1317] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:00:54 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:00:57.699522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bac69e-26df-4549-be6d-d108d6274bea None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:00:57.700656 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bac69e-26df-4549-be6d-d108d6274bea None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:00:57.700937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bac69e-26df-4549-be6d-d108d6274bea None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:00:57.702299 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 646/1318] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:00:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:00:57.707363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9dd9e-b297-48c6-9b5f-98d121ee7a78 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:00:57.707765 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9dd9e-b297-48c6-9b5f-98d121ee7a78 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:00:57.707903 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a9dd9e-b297-48c6-9b5f-98d121ee7a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:00:57.771926 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-e0a9dd9e-b297-48c6-9b5f-98d121ee7a78 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:00:57.773117 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-e0a9dd9e-b297-48c6-9b5f-98d121ee7a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:00:57.866202 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 673/1319] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:00:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:00:58.153307 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bac69e-26df-4549-be6d-d108d6274bea None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:00:58.154887 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-73bac69e-26df-4549-be6d-d108d6274bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:00:58.266486 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bac69e-26df-4549-be6d-d108d6274bea None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:00:58.337554 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca66ab2-d764-4242-9f88-68bae4486d89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:00:58.338155 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca66ab2-d764-4242-9f88-68bae4486d89 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:00:58.338431 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca66ab2-d764-4242-9f88-68bae4486d89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:00:58.338697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca66ab2-d764-4242-9f88-68bae4486d89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:00:58.340341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-5ca66ab2-d764-4242-9f88-68bae4486d89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:00:58.347831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:00:58.445592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca66ab2-d764-4242-9f88-68bae4486d89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:00:58.447051 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca66ab2-d764-4242-9f88-68bae4486d89 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:00:58.465019 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 647/1320] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:00:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:01:01.000755 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfcc09-627c-4e0c-9fbd-0376694aa1e2 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:01.001338 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfcc09-627c-4e0c-9fbd-0376694aa1e2 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:01.001338 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfcc09-627c-4e0c-9fbd-0376694aa1e2 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:01.002658 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 674/1321] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:01.007827 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c86bd-13e3-48e7-9783-3cfe31c94093 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:01.008256 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c86bd-13e3-48e7-9783-3cfe31c94093 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:01.008452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c86bd-13e3-48e7-9783-3cfe31c94093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:01.073746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-353c86bd-13e3-48e7-9783-3cfe31c94093 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:01.075557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-353c86bd-13e3-48e7-9783-3cfe31c94093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:01.132586 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:01.173852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 648/1322] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:01:01.470763 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a42e5-f846-4518-ba1c-b5b0ce10f080 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:01.471105 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a42e5-f846-4518-ba1c-b5b0ce10f080 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:01.471384 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a42e5-f846-4518-ba1c-b5b0ce10f080 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:01.472761 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 675/1323] 10.222.0.36 () {54 vars in 965 bytes} [Fri Sep 2 09:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:01.478537 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d2bdd-cbd4-46c0-924a-295fed4d51cd None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:01.478845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d2bdd-cbd4-46c0-924a-295fed4d51cd None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:01.479150 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d2bdd-cbd4-46c0-924a-295fed4d51cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:01.554891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-885d2bdd-cbd4-46c0-924a-295fed4d51cd None None] MFA Rules not processed for user `07a825fddbc04f918bcd631976038cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:01.556118 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-885d2bdd-cbd4-46c0-924a-295fed4d51cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:01.633666 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 649/1324] 10.222.0.36 () {58 vars in 1062 bytes} [Fri Sep 2 09:01:01 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:01:01.642489 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a42e5-f846-4518-ba1c-b5b0ce10f080 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:01:01.644035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-155a42e5-f846-4518-ba1c-b5b0ce10f080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:01.663198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:01.750008 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a42e5-f846-4518-ba1c-b5b0ce10f080 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:01:01.822454 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c3018-cc48-45d7-94bc-aabb950fbad1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:01:01.823055 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c3018-cc48-45d7-94bc-aabb950fbad1 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:01.823363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c3018-cc48-45d7-94bc-aabb950fbad1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:01.823636 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c3018-cc48-45d7-94bc-aabb950fbad1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:01.825292 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-466c3018-cc48-45d7-94bc-aabb950fbad1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:01.895054 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:01.932736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c3018-cc48-45d7-94bc-aabb950fbad1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:01:01.934203 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c3018-cc48-45d7-94bc-aabb950fbad1 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:01:01.952209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 676/1325] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:01:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:01:04.118399 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ee731-02cc-4521-b18b-06a089322a2b None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:04.118927 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ee731-02cc-4521-b18b-06a089322a2b None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:04.118927 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ee731-02cc-4521-b18b-06a089322a2b None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:04.120308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 650/1326] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:04.125955 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555c700-e92e-4fa5-8a03-afa54e14688c None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:04.126388 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555c700-e92e-4fa5-8a03-afa54e14688c None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:04.126528 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555c700-e92e-4fa5-8a03-afa54e14688c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:04.192906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-c555c700-e92e-4fa5-8a03-afa54e14688c None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:04.194098 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c555c700-e92e-4fa5-8a03-afa54e14688c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:04.299397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 677/1327] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:01:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:01:04.578629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ee731-02cc-4521-b18b-06a089322a2b None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:01:04.580307 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-af7ee731-02cc-4521-b18b-06a089322a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:04.639725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:04.684701 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ee731-02cc-4521-b18b-06a089322a2b None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:01:04.757999 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d96d9-cb31-4893-92e7-96b72c5f479a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:01:04.758611 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d96d9-cb31-4893-92e7-96b72c5f479a service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:04.758891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d96d9-cb31-4893-92e7-96b72c5f479a service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:04.759195 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d96d9-cb31-4893-92e7-96b72c5f479a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:04.760785 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-f86d96d9-cb31-4893-92e7-96b72c5f479a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:04.867518 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:04.870153 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d96d9-cb31-4893-92e7-96b72c5f479a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:01:04.872029 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d96d9-cb31-4893-92e7-96b72c5f479a service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:01:04.889968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 651/1328] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:01:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:01:06.848491 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c0e84-11c8-4346-b25d-1f783327f3e9 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:06.848965 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c0e84-11c8-4346-b25d-1f783327f3e9 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:06.849055 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c0e84-11c8-4346-b25d-1f783327f3e9 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:06.850636 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 678/1329] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:06.865233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41980da-15b3-4873-83cc-4c8e842afbca None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:06.865720 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41980da-15b3-4873-83cc-4c8e842afbca None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:06.865828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41980da-15b3-4873-83cc-4c8e842afbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:06.941244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-c41980da-15b3-4873-83cc-4c8e842afbca None None] MFA Rules not processed for user `9b90e57a93b44d24bc368ab2e96b81d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:06.942467 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c41980da-15b3-4873-83cc-4c8e842afbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:07.041108 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 652/1330] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:01:06 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:01:07.329110 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c0e84-11c8-4346-b25d-1f783327f3e9 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:01:07.330657 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8e1c0e84-11c8-4346-b25d-1f783327f3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:07.361793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:07.434944 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c0e84-11c8-4346-b25d-1f783327f3e9 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:01:07.509987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01e2c3-4891-4f3f-bad0-7e3363fff2e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:01:07.510764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01e2c3-4891-4f3f-bad0-7e3363fff2e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:07.510764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01e2c3-4891-4f3f-bad0-7e3363fff2e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:07.511127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01e2c3-4891-4f3f-bad0-7e3363fff2e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:07.512659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-2b01e2c3-4891-4f3f-bad0-7e3363fff2e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:07.593249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:07.627578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01e2c3-4891-4f3f-bad0-7e3363fff2e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:01:07.629026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01e2c3-4891-4f3f-bad0-7e3363fff2e8 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:01:07.648102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 679/1331] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:01:07 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:01:09.960765 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdac06-b924-41df-a0e3-a97a78598846 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:09.961346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdac06-b924-41df-a0e3-a97a78598846 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:09.961346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdac06-b924-41df-a0e3-a97a78598846 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:09.962648 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 653/1332] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:09.967831 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe45c3c-cbb1-4944-9739-fb417f55ee12 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:09.968244 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe45c3c-cbb1-4944-9739-fb417f55ee12 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:09.968372 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe45c3c-cbb1-4944-9739-fb417f55ee12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:10.035602 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-fbe45c3c-cbb1-4944-9739-fb417f55ee12 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:10.037527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-fbe45c3c-cbb1-4944-9739-fb417f55ee12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:10.137252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 680/1333] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:01:10.417159 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cdac06-b924-41df-a0e3-a97a78598846 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:01:10.418737 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-01cdac06-b924-41df-a0e3-a97a78598846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:10.488211 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:10.522506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cdac06-b924-41df-a0e3-a97a78598846 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:01:10.593905 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c309ef6-0230-4e01-96ee-133a4566b177 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:01:10.594510 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c309ef6-0230-4e01-96ee-133a4566b177 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:10.594790 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c309ef6-0230-4e01-96ee-133a4566b177 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:10.595061 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c309ef6-0230-4e01-96ee-133a4566b177 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:10.596698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7c309ef6-0230-4e01-96ee-133a4566b177 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:10.704018 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c309ef6-0230-4e01-96ee-133a4566b177 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:01:10.705377 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c309ef6-0230-4e01-96ee-133a4566b177 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:01:10.715177 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:10.724002 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 654/1334] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:01:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:01:12.658147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd5779-f111-4055-b95c-057f4295115f None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:12.658659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd5779-f111-4055-b95c-057f4295115f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:12.658659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbd5779-f111-4055-b95c-057f4295115f None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:12.659993 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 681/1335] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:12.665169 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ce79c-344c-4c82-8c2b-9fa7c991324f None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:12.665616 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ce79c-344c-4c82-8c2b-9fa7c991324f None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:12.665728 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ce79c-344c-4c82-8c2b-9fa7c991324f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:12.729980 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-250ce79c-344c-4c82-8c2b-9fa7c991324f None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:12.731239 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-250ce79c-344c-4c82-8c2b-9fa7c991324f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:12.827228 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 655/1336] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:01:13.109374 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd5779-f111-4055-b95c-057f4295115f None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:01:13.110899 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-7dbd5779-f111-4055-b95c-057f4295115f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:13.150705 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:13.213597 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbd5779-f111-4055-b95c-057f4295115f None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:01:13.284164 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4fa594-0c47-4833-945b-071dd0aea620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:01:13.284761 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fa594-0c47-4833-945b-071dd0aea620 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:13.285033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fa594-0c47-4833-945b-071dd0aea620 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:13.285298 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4fa594-0c47-4833-945b-071dd0aea620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:13.286857 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-1f4fa594-0c47-4833-945b-071dd0aea620 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:13.373312 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:13.398544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4fa594-0c47-4833-945b-071dd0aea620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:01:13.400004 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4fa594-0c47-4833-945b-071dd0aea620 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:01:13.417609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 682/1337] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:01:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:01:15.678592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18c23f-dbc9-4114-a9bf-273d27508fa5 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:15.679142 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18c23f-dbc9-4114-a9bf-273d27508fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:15.679142 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18c23f-dbc9-4114-a9bf-273d27508fa5 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:15.680457 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 656/1338] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:15.685652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba1157-1cb1-4f28-873e-6d0947dc1cbd None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:15.686307 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba1157-1cb1-4f28-873e-6d0947dc1cbd None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:15.686451 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba1157-1cb1-4f28-873e-6d0947dc1cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:15.751889 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-c4ba1157-1cb1-4f28-873e-6d0947dc1cbd None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:15.753129 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c4ba1157-1cb1-4f28-873e-6d0947dc1cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:15.848848 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 683/1339] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:01:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:01:16.131342 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18c23f-dbc9-4114-a9bf-273d27508fa5 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:01:16.132892 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1f18c23f-dbc9-4114-a9bf-273d27508fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:16.219160 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:16.247221 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18c23f-dbc9-4114-a9bf-273d27508fa5 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:01:16.317978 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca94d17-1ace-4a02-b83a-113faeda6adf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:01:16.318577 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca94d17-1ace-4a02-b83a-113faeda6adf service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:16.318852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca94d17-1ace-4a02-b83a-113faeda6adf service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:16.319146 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca94d17-1ace-4a02-b83a-113faeda6adf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:16.320722 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-4ca94d17-1ace-4a02-b83a-113faeda6adf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:16.433238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca94d17-1ace-4a02-b83a-113faeda6adf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:01:16.435160 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca94d17-1ace-4a02-b83a-113faeda6adf service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:01:16.453183 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 657/1340] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:01:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:01:20.294084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6618e-d985-4101-9bce-b1ef87e1bab4 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:20.294633 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6618e-d985-4101-9bce-b1ef87e1bab4 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:20.294633 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f6618e-d985-4101-9bce-b1ef87e1bab4 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:20.296076 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 684/1341] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:01:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:20.301024 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e00953-51af-4459-a888-020ad9e0dc74 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:20.301432 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e00953-51af-4459-a888-020ad9e0dc74 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:20.301586 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e00953-51af-4459-a888-020ad9e0dc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:20.306999 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:20.380077 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-76e00953-51af-4459-a888-020ad9e0dc74 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:20.381279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-76e00953-51af-4459-a888-020ad9e0dc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:20.476353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 658/1342] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:01:20.758267 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6618e-d985-4101-9bce-b1ef87e1bab4 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:01:20.759896 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-90f6618e-d985-4101-9bce-b1ef87e1bab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:20.808192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:20.864380 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f6618e-d985-4101-9bce-b1ef87e1bab4 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:01:20.937187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e1d3fc-6630-4b7d-a8a5-a60b57903904 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:01:20.937789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1d3fc-6630-4b7d-a8a5-a60b57903904 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:20.938064 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1d3fc-6630-4b7d-a8a5-a60b57903904 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:20.938331 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e1d3fc-6630-4b7d-a8a5-a60b57903904 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:20.939932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-65e1d3fc-6630-4b7d-a8a5-a60b57903904 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:21.033348 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:21.050084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1d3fc-6630-4b7d-a8a5-a60b57903904 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:01:21.051573 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e1d3fc-6630-4b7d-a8a5-a60b57903904 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:01:21.069863 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 685/1343] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:01:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:01:23.591029 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83cd4e-e982-4e14-9959-680d89c11600 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:23.591592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83cd4e-e982-4e14-9959-680d89c11600 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:23.591592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee83cd4e-e982-4e14-9959-680d89c11600 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:23.592950 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 659/1344] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:23.597997 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21380f-23d4-42cf-bd2c-7be24f6ebb3d None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:23.598435 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21380f-23d4-42cf-bd2c-7be24f6ebb3d None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:23.598558 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21380f-23d4-42cf-bd2c-7be24f6ebb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:23.664194 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-5c21380f-23d4-42cf-bd2c-7be24f6ebb3d None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:23.665298 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5c21380f-23d4-42cf-bd2c-7be24f6ebb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:23.761402 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 686/1345] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:01:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:01:24.039840 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83cd4e-e982-4e14-9959-680d89c11600 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:01:24.041412 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ee83cd4e-e982-4e14-9959-680d89c11600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:24.129758 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:24.146562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee83cd4e-e982-4e14-9959-680d89c11600 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:01:24.222637 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84efa59-b0da-4ee1-8673-5e07992bcce7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:01:24.223288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84efa59-b0da-4ee1-8673-5e07992bcce7 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:24.223563 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84efa59-b0da-4ee1-8673-5e07992bcce7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:24.223824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84efa59-b0da-4ee1-8673-5e07992bcce7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:24.225515 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-f84efa59-b0da-4ee1-8673-5e07992bcce7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:24.337454 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84efa59-b0da-4ee1-8673-5e07992bcce7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:01:24.338817 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84efa59-b0da-4ee1-8673-5e07992bcce7 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:01:24.359140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 660/1346] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:01:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:01:28.250729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d4505-ed05-4f8a-81cb-995a0e8e932d None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:28.251707 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d4505-ed05-4f8a-81cb-995a0e8e932d None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:28.252213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d4505-ed05-4f8a-81cb-995a0e8e932d None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:28.254119 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 687/1347] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:01:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:28.259352 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96464876-537c-4ae4-8f47-fc50c4cefba1 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:28.259863 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96464876-537c-4ae4-8f47-fc50c4cefba1 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:28.259970 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96464876-537c-4ae4-8f47-fc50c4cefba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:28.276899 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:28.331364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-96464876-537c-4ae4-8f47-fc50c4cefba1 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:28.332584 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-96464876-537c-4ae4-8f47-fc50c4cefba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:28.434591 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 661/1348] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:01:28.717780 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d4505-ed05-4f8a-81cb-995a0e8e932d None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:01:28.719525 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-101d4505-ed05-4f8a-81cb-995a0e8e932d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:28.782446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:28.829648 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d4505-ed05-4f8a-81cb-995a0e8e932d None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:01:28.907091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b149fc8a-e2aa-4860-8883-aae7c359dfde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:01:28.907714 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149fc8a-e2aa-4860-8883-aae7c359dfde service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:28.907994 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149fc8a-e2aa-4860-8883-aae7c359dfde service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:28.908266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149fc8a-e2aa-4860-8883-aae7c359dfde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:28.909879 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-b149fc8a-e2aa-4860-8883-aae7c359dfde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:29.014911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:29.019270 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b149fc8a-e2aa-4860-8883-aae7c359dfde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:01:29.020704 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b149fc8a-e2aa-4860-8883-aae7c359dfde service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:01:29.038684 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 688/1349] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:01:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:01:33.926220 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44638845-697c-4389-a808-43569fd2e3ed None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:33.926648 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44638845-697c-4389-a808-43569fd2e3ed None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:33.926747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44638845-697c-4389-a808-43569fd2e3ed None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:33.929151 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 662/1350] 10.222.0.36 () {54 vars in 921 bytes} [Fri Sep 2 09:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:33.933185 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889e2a9-1a35-457c-9dae-95c7845fd2ef None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:33.933610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889e2a9-1a35-457c-9dae-95c7845fd2ef None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:33.933831 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889e2a9-1a35-457c-9dae-95c7845fd2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:34.035669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-d889e2a9-1a35-457c-9dae-95c7845fd2ef None None] MFA Rules not processed for user `6a0f0fdbbd20492d8c35ec4b73404632`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:34.037238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-d889e2a9-1a35-457c-9dae-95c7845fd2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:34.202029 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 689/1351] 10.222.0.36 () {58 vars in 1018 bytes} [Fri Sep 2 09:01:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:01:38.432093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=59 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:38.505875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce30119-9019-459f-b47e-52569f4616d6 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:38.506572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce30119-9019-459f-b47e-52569f4616d6 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:38.507101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce30119-9019-459f-b47e-52569f4616d6 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:38.508938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 663/1352] 10.222.0.36 () {54 vars in 921 bytes} [Fri Sep 2 09:01:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:38.514519 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b197c9-0440-4956-b7d0-0dab52db65b0 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:38.515584 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b197c9-0440-4956-b7d0-0dab52db65b0 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:38.516066 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b197c9-0440-4956-b7d0-0dab52db65b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:38.573851 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:38.598910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-a5b197c9-0440-4956-b7d0-0dab52db65b0 None None] MFA Rules not processed for user `6a0f0fdbbd20492d8c35ec4b73404632`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:38.600409 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a5b197c9-0440-4956-b7d0-0dab52db65b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:38.729778 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 690/1353] 10.222.0.36 () {58 vars in 1018 bytes} [Fri Sep 2 09:01:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:01:39.916429 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70d9e3-b8c6-46ff-92d6-2dddb82e8599 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:39.916957 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70d9e3-b8c6-46ff-92d6-2dddb82e8599 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:39.916957 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db70d9e3-b8c6-46ff-92d6-2dddb82e8599 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:39.918837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 664/1354] 10.222.0.36 () {54 vars in 921 bytes} [Fri Sep 2 09:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:39.923879 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff3414-7adb-461b-a7a4-3079795a34c6 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:39.924311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff3414-7adb-461b-a7a4-3079795a34c6 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:39.924568 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff3414-7adb-461b-a7a4-3079795a34c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:40.001894 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-f3ff3414-7adb-461b-a7a4-3079795a34c6 None None] MFA Rules not processed for user `6a0f0fdbbd20492d8c35ec4b73404632`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:40.003103 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f3ff3414-7adb-461b-a7a4-3079795a34c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:40.042741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:40.109984 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 691/1355] 10.222.0.36 () {58 vars in 1018 bytes} [Fri Sep 2 09:01:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:01:43.433279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a1998-a514-4274-8634-efa3ee0243fb None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:43.434360 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a1998-a514-4274-8634-efa3ee0243fb None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:43.434360 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a1998-a514-4274-8634-efa3ee0243fb None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:43.435838 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 665/1356] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:01:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:43.441095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df868ac5-1d32-4492-8a9c-87fef835009a None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:43.441524 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df868ac5-1d32-4492-8a9c-87fef835009a None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:43.441636 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df868ac5-1d32-4492-8a9c-87fef835009a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:43.531055 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-df868ac5-1d32-4492-8a9c-87fef835009a None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:43.532290 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-df868ac5-1d32-4492-8a9c-87fef835009a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:43.652204 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 692/1357] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:01:43.964709 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dee70-1787-4a85-a220-e0e20b00d7a4 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:43.965491 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dee70-1787-4a85-a220-e0e20b00d7a4 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:43.965981 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dee70-1787-4a85-a220-e0e20b00d7a4 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:43.967333 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 666/1358] 10.222.0.36 () {54 vars in 962 bytes} [Fri Sep 2 09:01:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:43.976850 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758a929-8427-4e74-b083-ce6fe88f7c9a None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:43.977250 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758a929-8427-4e74-b083-ce6fe88f7c9a None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:43.977680 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758a929-8427-4e74-b083-ce6fe88f7c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:44.005704 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:44.070055 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-3758a929-8427-4e74-b083-ce6fe88f7c9a None None] MFA Rules not processed for user `cce502a83f5343558c2136c0193e7d85`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:44.071711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-3758a929-8427-4e74-b083-ce6fe88f7c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:44.190694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 693/1359] 10.222.0.36 () {58 vars in 1059 bytes} [Fri Sep 2 09:01:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:01:44.198245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dee70-1787-4a85-a220-e0e20b00d7a4 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:01:44.200255 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-d67dee70-1787-4a85-a220-e0e20b00d7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:44.356883 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dee70-1787-4a85-a220-e0e20b00d7a4 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:01:44.441114 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:44.486292 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f611fa5-8aca-439f-834c-89491ae986c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:01:44.487381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f611fa5-8aca-439f-834c-89491ae986c6 service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:44.487672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f611fa5-8aca-439f-834c-89491ae986c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:44.487953 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f611fa5-8aca-439f-834c-89491ae986c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:44.489683 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7f611fa5-8aca-439f-834c-89491ae986c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:44.586181 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f611fa5-8aca-439f-834c-89491ae986c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:01:44.587809 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f611fa5-8aca-439f-834c-89491ae986c6 service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:01:44.607866 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 667/1360] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:01:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:01:46.765930 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341e482-5db2-424e-8fe4-94793f63efc0 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:46.766481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341e482-5db2-424e-8fe4-94793f63efc0 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:46.766481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341e482-5db2-424e-8fe4-94793f63efc0 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:46.768021 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 694/1361] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:46.773034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db0169-284a-4380-890d-d859735a7165 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:46.773506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db0169-284a-4380-890d-d859735a7165 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:46.773598 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db0169-284a-4380-890d-d859735a7165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:46.846810 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-b0db0169-284a-4380-890d-d859735a7165 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:46.848322 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b0db0169-284a-4380-890d-d859735a7165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:46.890234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:46.962875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 668/1362] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:01:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:01:47.251132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341e482-5db2-424e-8fe4-94793f63efc0 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:01:47.251132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-7341e482-5db2-424e-8fe4-94793f63efc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:47.324612 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:47.421934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341e482-5db2-424e-8fe4-94793f63efc0 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:01:47.549554 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d35be-1f13-468c-ae3f-39269e3fcdd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:01:47.550510 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d35be-1f13-468c-ae3f-39269e3fcdd6 service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:47.550788 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d35be-1f13-468c-ae3f-39269e3fcdd6 service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:47.551265 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d35be-1f13-468c-ae3f-39269e3fcdd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:47.552889 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-259d35be-1f13-468c-ae3f-39269e3fcdd6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:47.580785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:47.655862 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d35be-1f13-468c-ae3f-39269e3fcdd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:01:47.657457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d35be-1f13-468c-ae3f-39269e3fcdd6 service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:01:47.677561 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 695/1363] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:01:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:01:49.951179 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2782f2-8477-48c8-b260-578b0f569f0a None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:49.951739 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2782f2-8477-48c8-b260-578b0f569f0a None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:49.951739 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2782f2-8477-48c8-b260-578b0f569f0a None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:49.953092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 669/1364] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:49.958554 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190780d-fc8a-4b17-8f0b-ab6ad18f7a4b None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:49.959047 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190780d-fc8a-4b17-8f0b-ab6ad18f7a4b None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:49.959189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190780d-fc8a-4b17-8f0b-ab6ad18f7a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:50.036683 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-6190780d-fc8a-4b17-8f0b-ab6ad18f7a4b None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:50.037917 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6190780d-fc8a-4b17-8f0b-ab6ad18f7a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:50.142455 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:50.145685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 696/1365] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:01:50.440567 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adf40b-f930-4425-ace5-8ad206b359ed None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:50.440851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adf40b-f930-4425-ace5-8ad206b359ed None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:50.441304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adf40b-f930-4425-ace5-8ad206b359ed None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:50.442808 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 670/1366] 10.222.0.36 () {54 vars in 962 bytes} [Fri Sep 2 09:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:50.448340 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcad5e4-39fd-4ae1-93cf-2ae74468a894 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:50.448841 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcad5e4-39fd-4ae1-93cf-2ae74468a894 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:50.449920 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcad5e4-39fd-4ae1-93cf-2ae74468a894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:50.538091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-1bcad5e4-39fd-4ae1-93cf-2ae74468a894 None None] MFA Rules not processed for user `cce502a83f5343558c2136c0193e7d85`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:50.539401 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-1bcad5e4-39fd-4ae1-93cf-2ae74468a894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:50.665833 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 697/1367] 10.222.0.36 () {58 vars in 1059 bytes} [Fri Sep 2 09:01:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:01:50.673924 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adf40b-f930-4425-ace5-8ad206b359ed None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:01:50.675599 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-77adf40b-f930-4425-ace5-8ad206b359ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:50.834736 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77adf40b-f930-4425-ace5-8ad206b359ed None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:01:50.856698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:50.962190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0b528-b62b-4543-9e40-e69238818f7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:01:50.962879 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0b528-b62b-4543-9e40-e69238818f7f service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:50.963145 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0b528-b62b-4543-9e40-e69238818f7f service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:50.963367 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0b528-b62b-4543-9e40-e69238818f7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:50.965000 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-f5c0b528-b62b-4543-9e40-e69238818f7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:51.059856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0b528-b62b-4543-9e40-e69238818f7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:01:51.061940 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0b528-b62b-4543-9e40-e69238818f7f service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:01:51.082791 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 671/1368] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:01:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:01:51.782539 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5c152-cdad-49b3-87c5-c25264288d9f None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:51.783105 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5c152-cdad-49b3-87c5-c25264288d9f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:51.783105 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5c152-cdad-49b3-87c5-c25264288d9f None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:51.784527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 698/1369] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:51.789778 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8782b3-6029-48c7-b955-6476a5fd96c1 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:51.790162 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8782b3-6029-48c7-b955-6476a5fd96c1 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:51.790528 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8782b3-6029-48c7-b955-6476a5fd96c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:51.825872 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:51.862000 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-2c8782b3-6029-48c7-b955-6476a5fd96c1 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:51.863372 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-2c8782b3-6029-48c7-b955-6476a5fd96c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:51.967123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 672/1370] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:01:52.542562 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e33d5-5454-4b7d-8633-d26039931fc2 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:52.542562 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e33d5-5454-4b7d-8633-d26039931fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:52.542562 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e33d5-5454-4b7d-8633-d26039931fc2 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:52.543904 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 699/1371] 10.222.0.36 () {54 vars in 964 bytes} [Fri Sep 2 09:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:52.549066 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f76b7-6c90-4ae1-990e-b245ca1a2833 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:52.549344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f76b7-6c90-4ae1-990e-b245ca1a2833 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:52.549603 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f76b7-6c90-4ae1-990e-b245ca1a2833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:52.629534 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-fb1f76b7-6c90-4ae1-990e-b245ca1a2833 None None] MFA Rules not processed for user `f8370135a78f414f9184c883ccb92767`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:52.630934 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-fb1f76b7-6c90-4ae1-990e-b245ca1a2833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:52.681474 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:52.713783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 673/1372] 10.222.0.36 () {58 vars in 1061 bytes} [Fri Sep 2 09:01:52 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:01:52.722401 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e33d5-5454-4b7d-8633-d26039931fc2 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:01:52.723946 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-753e33d5-5454-4b7d-8633-d26039931fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:52.781178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:52.841431 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e33d5-5454-4b7d-8633-d26039931fc2 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:01:52.918159 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4fabf-0136-479a-a59e-88c7c96da3fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:01:52.918648 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4fabf-0136-479a-a59e-88c7c96da3fe service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:52.918990 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4fabf-0136-479a-a59e-88c7c96da3fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:52.919339 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be4fabf-0136-479a-a59e-88c7c96da3fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:52.920806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-4be4fabf-0136-479a-a59e-88c7c96da3fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:53.024307 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:53.037656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4fabf-0136-479a-a59e-88c7c96da3fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:01:53.039184 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be4fabf-0136-479a-a59e-88c7c96da3fe service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:01:53.047704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c4ca6-9b51-4236-984a-31722bde990e None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:53.047704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c4ca6-9b51-4236-984a-31722bde990e None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:53.048785 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c4ca6-9b51-4236-984a-31722bde990e None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:53.050474 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 674/1373] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:53.054193 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a454756-badb-4152-98e2-dac772bca0ad None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:53.054773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a454756-badb-4152-98e2-dac772bca0ad None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:53.054773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a454756-badb-4152-98e2-dac772bca0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:53.061371 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 700/1374] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:01:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:01:53.132628 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-5a454756-badb-4152-98e2-dac772bca0ad None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:53.134062 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-5a454756-badb-4152-98e2-dac772bca0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:53.238355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 675/1375] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:01:53.524317 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4fabf-0136-479a-a59e-88c7c96da3fe service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:01:53.525502 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-4be4fabf-0136-479a-a59e-88c7c96da3fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:53.623384 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be4fabf-0136-479a-a59e-88c7c96da3fe service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:01:53.725502 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:53.739769 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bed9da-0dab-4010-b9de-1d8a3108fca3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:01:53.740378 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bed9da-0dab-4010-b9de-1d8a3108fca3 service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:53.740654 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bed9da-0dab-4010-b9de-1d8a3108fca3 service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:53.741015 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bed9da-0dab-4010-b9de-1d8a3108fca3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:53.742852 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c9bed9da-0dab-4010-b9de-1d8a3108fca3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:53.831272 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bed9da-0dab-4010-b9de-1d8a3108fca3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:01:53.832763 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bed9da-0dab-4010-b9de-1d8a3108fca3 service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:01:53.850731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 701/1376] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:01:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:01:55.788439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d580f7-0cee-4c13-9a56-4388c7071422 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:55.788945 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d580f7-0cee-4c13-9a56-4388c7071422 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:55.788945 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d580f7-0cee-4c13-9a56-4388c7071422 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:55.790243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 676/1377] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:55.796095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cacf97-a23e-4459-ad98-a493dd5c9d4e None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:55.796523 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cacf97-a23e-4459-ad98-a493dd5c9d4e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:55.796629 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cacf97-a23e-4459-ad98-a493dd5c9d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:55.864655 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-94cacf97-a23e-4459-ad98-a493dd5c9d4e None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:55.865848 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-94cacf97-a23e-4459-ad98-a493dd5c9d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:55.933350 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:55.973771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 702/1378] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:01:56.267501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d580f7-0cee-4c13-9a56-4388c7071422 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:01:56.269084 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-88d580f7-0cee-4c13-9a56-4388c7071422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:56.322302 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:56.429727 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d580f7-0cee-4c13-9a56-4388c7071422 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:01:56.591579 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14298c85-9b3b-456f-ad41-50d80accadc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:01:56.592435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14298c85-9b3b-456f-ad41-50d80accadc9 service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:56.592864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14298c85-9b3b-456f-ad41-50d80accadc9 service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:56.593282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14298c85-9b3b-456f-ad41-50d80accadc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:56.595151 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-14298c85-9b3b-456f-ad41-50d80accadc9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:56.610296 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:56.696852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14298c85-9b3b-456f-ad41-50d80accadc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:01:56.698518 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14298c85-9b3b-456f-ad41-50d80accadc9 service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:01:56.717432 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 677/1379] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:01:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:01:58.662731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213bddb-0de9-47ba-b137-a5b522b21fa8 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:58.663300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213bddb-0de9-47ba-b137-a5b522b21fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:58.663300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213bddb-0de9-47ba-b137-a5b522b21fa8 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:58.664724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 703/1380] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:01:58.670470 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace40a6-12c8-42f7-ad13-063f54daeb51 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:58.670907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace40a6-12c8-42f7-ad13-063f54daeb51 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:58.670999 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace40a6-12c8-42f7-ad13-063f54daeb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:58.738990 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-5ace40a6-12c8-42f7-ad13-063f54daeb51 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:01:58.740206 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-5ace40a6-12c8-42f7-ad13-063f54daeb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:58.831003 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:58.845898 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 678/1381] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:01:59.134618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213bddb-0de9-47ba-b137-a5b522b21fa8 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:01:59.136159 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-1213bddb-0de9-47ba-b137-a5b522b21fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:59.304526 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213bddb-0de9-47ba-b137-a5b522b21fa8 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:01:59.367244 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:01:59.445336 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa2539-257a-4c3d-8410-5d5954d6b65e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:01:59.445907 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa2539-257a-4c3d-8410-5d5954d6b65e service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:01:59.446295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa2539-257a-4c3d-8410-5d5954d6b65e service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:01:59.446582 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa2539-257a-4c3d-8410-5d5954d6b65e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:01:59.448171 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6bfa2539-257a-4c3d-8410-5d5954d6b65e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:01:59.544699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa2539-257a-4c3d-8410-5d5954d6b65e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:01:59.546290 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa2539-257a-4c3d-8410-5d5954d6b65e service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:01:59.567115 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 704/1382] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:01:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:02:01.526681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029afeb0-24c1-4662-a506-b60ce5343eb9 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:01.527331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029afeb0-24c1-4662-a506-b60ce5343eb9 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:01.528173 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029afeb0-24c1-4662-a506-b60ce5343eb9 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:01.529894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 679/1383] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:02:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:01.537563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d3846-2fa2-4da1-b19b-e8fd810e65c0 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:01.538004 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d3846-2fa2-4da1-b19b-e8fd810e65c0 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:01.538160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d3846-2fa2-4da1-b19b-e8fd810e65c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:01.607647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-669d3846-2fa2-4da1-b19b-e8fd810e65c0 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:01.608819 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-669d3846-2fa2-4da1-b19b-e8fd810e65c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:01.611011 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:01.719584 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 705/1384] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:02.026852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029afeb0-24c1-4662-a506-b60ce5343eb9 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:02:02.028497 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-029afeb0-24c1-4662-a506-b60ce5343eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:02.185568 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029afeb0-24c1-4662-a506-b60ce5343eb9 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:02:02.255155 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:02.306477 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d2d6b-2e37-40fc-b152-9f462f0a38a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:02:02.307093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d2d6b-2e37-40fc-b152-9f462f0a38a0 service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:02.307372 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d2d6b-2e37-40fc-b152-9f462f0a38a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:02.307643 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d2d6b-2e37-40fc-b152-9f462f0a38a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:02.309331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-192d2d6b-2e37-40fc-b152-9f462f0a38a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:02.400040 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d2d6b-2e37-40fc-b152-9f462f0a38a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:02:02.401621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d2d6b-2e37-40fc-b152-9f462f0a38a0 service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:02:02.419935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 680/1385] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:02:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:02:04.424529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dad5d-1371-4d11-b05b-6cf836a2d36e None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:04.425084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dad5d-1371-4d11-b05b-6cf836a2d36e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:04.425213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10dad5d-1371-4d11-b05b-6cf836a2d36e None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:04.426546 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 706/1386] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:04.432855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8acd1-baca-43ab-96f6-7ce6ebe1c9f2 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:04.433408 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8acd1-baca-43ab-96f6-7ce6ebe1c9f2 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:04.433408 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8acd1-baca-43ab-96f6-7ce6ebe1c9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:04.504584 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-61c8acd1-baca-43ab-96f6-7ce6ebe1c9f2 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:04.505895 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-61c8acd1-baca-43ab-96f6-7ce6ebe1c9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:04.533329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:04.620021 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 681/1387] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:04.981708 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10dad5d-1371-4d11-b05b-6cf836a2d36e None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:02:04.983332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-e10dad5d-1371-4d11-b05b-6cf836a2d36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:05.123686 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:05.140898 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10dad5d-1371-4d11-b05b-6cf836a2d36e None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:02:05.258211 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4c878-3134-4b8a-be37-68fc187c7616 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:02:05.258760 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4c878-3134-4b8a-be37-68fc187c7616 service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:05.259047 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4c878-3134-4b8a-be37-68fc187c7616 service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:05.259363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4c878-3134-4b8a-be37-68fc187c7616 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:05.261063 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-89b4c878-3134-4b8a-be37-68fc187c7616 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:05.357034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4c878-3134-4b8a-be37-68fc187c7616 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:02:05.358649 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4c878-3134-4b8a-be37-68fc187c7616 service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:02:05.363722 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:05.378210 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 707/1388] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:02:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:02:07.325315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7bc59-7c81-49ff-ad4e-a9c9bcdde57e None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:07.325860 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7bc59-7c81-49ff-ad4e-a9c9bcdde57e None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:07.325860 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7bc59-7c81-49ff-ad4e-a9c9bcdde57e None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:07.327318 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 682/1389] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:07.332746 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba675158-fe84-4203-8fc4-35bbbefc276a None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:07.333162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba675158-fe84-4203-8fc4-35bbbefc276a None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:07.333286 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba675158-fe84-4203-8fc4-35bbbefc276a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:07.399928 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-ba675158-fe84-4203-8fc4-35bbbefc276a None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:07.401135 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-ba675158-fe84-4203-8fc4-35bbbefc276a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:07.498269 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 708/1390] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:07.786828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7bc59-7c81-49ff-ad4e-a9c9bcdde57e None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:02:07.788406 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-75a7bc59-7c81-49ff-ad4e-a9c9bcdde57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:07.930791 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7bc59-7c81-49ff-ad4e-a9c9bcdde57e None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:02:07.940512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:08.012676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594601f-fb8e-45fa-84f6-e2dca7b15391 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:08.012969 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594601f-fb8e-45fa-84f6-e2dca7b15391 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:08.013678 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594601f-fb8e-45fa-84f6-e2dca7b15391 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:08.015330 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 709/1391] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:02:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:08.020042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1284130-aaca-43b7-8302-6764ccb452a9 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:08.020325 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1284130-aaca-43b7-8302-6764ccb452a9 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:08.020597 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1284130-aaca-43b7-8302-6764ccb452a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:08.042565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04477075-34eb-48ee-b019-2b1d0b54feaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:02:08.043282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04477075-34eb-48ee-b019-2b1d0b54feaf service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:08.043560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04477075-34eb-48ee-b019-2b1d0b54feaf service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:08.043828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04477075-34eb-48ee-b019-2b1d0b54feaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:08.045658 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-04477075-34eb-48ee-b019-2b1d0b54feaf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:08.086716 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-f1284130-aaca-43b7-8302-6764ccb452a9 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:08.087935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f1284130-aaca-43b7-8302-6764ccb452a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:08.109097 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:08.134747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04477075-34eb-48ee-b019-2b1d0b54feaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:02:08.136364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04477075-34eb-48ee-b019-2b1d0b54feaf service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:02:08.156486 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 683/1392] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:02:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:02:08.191591 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 710/1393] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:02:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:02:08.491820 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93262c3-baf9-423b-9028-f97edfadc8aa None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:08.492313 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93262c3-baf9-423b-9028-f97edfadc8aa None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:08.492409 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93262c3-baf9-423b-9028-f97edfadc8aa None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:08.493778 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 684/1394] 10.222.0.36 () {54 vars in 961 bytes} [Fri Sep 2 09:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:08.499049 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001db0e-1647-4006-a627-081712433e77 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:08.499501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001db0e-1647-4006-a627-081712433e77 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:08.499618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001db0e-1647-4006-a627-081712433e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:08.591511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-a001db0e-1647-4006-a627-081712433e77 None None] MFA Rules not processed for user `2474d89f10764afbbc5e740dccaae4df`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:08.593142 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a001db0e-1647-4006-a627-081712433e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:08.670366 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 711/1395] 10.222.0.36 () {58 vars in 1058 bytes} [Fri Sep 2 09:02:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:08.677837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93262c3-baf9-423b-9028-f97edfadc8aa None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:02:08.679394 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-d93262c3-baf9-423b-9028-f97edfadc8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:08.693377 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:08.782775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93262c3-baf9-423b-9028-f97edfadc8aa None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:02:08.856368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4100958-3e29-4295-87bf-ee66df6cd855 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2474d89f10764afbbc5e740dccaae4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:02:08.856979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4100958-3e29-4295-87bf-ee66df6cd855 service glance] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:08.857260 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4100958-3e29-4295-87bf-ee66df6cd855 service glance] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:08.857532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4100958-3e29-4295-87bf-ee66df6cd855 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:08.859178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-f4100958-3e29-4295-87bf-ee66df6cd855 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:08.925614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:08.967770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4100958-3e29-4295-87bf-ee66df6cd855 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:02:08.969180 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4100958-3e29-4295-87bf-ee66df6cd855 service glance] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:02:08.988606 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 685/1396] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:02:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:02:09.717401 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84224978-4020-4b5e-9166-05c612f0a26e None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:09.717929 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84224978-4020-4b5e-9166-05c612f0a26e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:09.717929 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84224978-4020-4b5e-9166-05c612f0a26e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:09.738295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:09.789212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-84224978-4020-4b5e-9166-05c612f0a26e None None] MFA Rules not processed for user `76ebecce283f491e82a961d7f739878f`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:09.790423 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-84224978-4020-4b5e-9166-05c612f0a26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:09.876750 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 712/1397] 10.222.0.36 () {58 vars in 1009 bytes} [Fri Sep 2 09:02:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:09.918383 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed72572-f4e1-4961-8883-1cc91aec2a0d None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:09.918860 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed72572-f4e1-4961-8883-1cc91aec2a0d None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:09.918961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed72572-f4e1-4961-8883-1cc91aec2a0d None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:09.920331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 686/1398] 10.222.0.36 () {54 vars in 943 bytes} [Fri Sep 2 09:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:09.926359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025fa42-873c-47ef-bf9b-3551d22ae43c None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:09.926767 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025fa42-873c-47ef-bf9b-3551d22ae43c None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:09.927054 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025fa42-873c-47ef-bf9b-3551d22ae43c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:10.004177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-b025fa42-873c-47ef-bf9b-3551d22ae43c None None] MFA Rules not processed for user `92318948dcf1407b9fc8aa5c199787ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:10.005379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-b025fa42-873c-47ef-bf9b-3551d22ae43c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:10.053596 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:10.113229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 713/1399] 10.222.0.36 () {58 vars in 1040 bytes} [Fri Sep 2 09:02:09 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:10.119232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde2ae5-97ad-4b7b-91bd-dcd6ef2ceba9 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:10.119515 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde2ae5-97ad-4b7b-91bd-dcd6ef2ceba9 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:10.119838 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde2ae5-97ad-4b7b-91bd-dcd6ef2ceba9 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:10.121597 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 687/1400] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:10.123882 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025fa42-873c-47ef-bf9b-3551d22ae43c None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:02:10.126652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e2031-cd91-4a96-a093-87a21f01ac2e None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:10.126935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e2031-cd91-4a96-a093-87a21f01ac2e None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:10.127246 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e2031-cd91-4a96-a093-87a21f01ac2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:10.133576 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025fa42-873c-47ef-bf9b-3551d22ae43c None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:02:10.193718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-f33e2031-cd91-4a96-a093-87a21f01ac2e None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:10.194850 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-f33e2031-cd91-4a96-a093-87a21f01ac2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:10.244128 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c204c-7d8d-4871-9c86-8b91027311a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92318948dcf1407b9fc8aa5c199787ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:02:10.244739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c204c-7d8d-4871-9c86-8b91027311a8 service swift] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:10.245014 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c204c-7d8d-4871-9c86-8b91027311a8 service swift] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:10.245285 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c204c-7d8d-4871-9c86-8b91027311a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:10.247033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-967c204c-7d8d-4871-9c86-8b91027311a8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:10.301411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 688/1401] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:02:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:02:10.305495 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:10.341156 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c204c-7d8d-4871-9c86-8b91027311a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:02:10.342760 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c204c-7d8d-4871-9c86-8b91027311a8 service swift] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:02:10.353359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 714/1402] 10.222.0.36 () {60 vars in 1384 bytes} [Fri Sep 2 09:02:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:02:10.592828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e2031-cd91-4a96-a093-87a21f01ac2e None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:02:10.594057 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-f33e2031-cd91-4a96-a093-87a21f01ac2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:10.623000 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:10.722470 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e2031-cd91-4a96-a093-87a21f01ac2e None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:02:10.859722 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9751b55-4852-4c67-bfd5-0456537c77da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:02:10.860456 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9751b55-4852-4c67-bfd5-0456537c77da service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:10.860801 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9751b55-4852-4c67-bfd5-0456537c77da service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:10.861144 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9751b55-4852-4c67-bfd5-0456537c77da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:10.862818 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-f9751b55-4852-4c67-bfd5-0456537c77da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:10.881647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:10.958227 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9751b55-4852-4c67-bfd5-0456537c77da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:02:10.959898 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9751b55-4852-4c67-bfd5-0456537c77da service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:02:10.982055 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 689/1403] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:02:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:02:12.947800 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b101ec-a64f-434c-b81b-5caa9f23024b None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:12.948346 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b101ec-a64f-434c-b81b-5caa9f23024b None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:12.948429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b101ec-a64f-434c-b81b-5caa9f23024b None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:12.949938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 715/1404] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:12.960845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ea73e-9ebd-42c4-b04e-335dd614ecf4 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:12.960845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ea73e-9ebd-42c4-b04e-335dd614ecf4 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:12.960845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ea73e-9ebd-42c4-b04e-335dd614ecf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:13.034328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-8f4ea73e-9ebd-42c4-b04e-335dd614ecf4 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:13.035580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-8f4ea73e-9ebd-42c4-b04e-335dd614ecf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:13.136744 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:13.147111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 690/1405] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:13.439302 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b101ec-a64f-434c-b81b-5caa9f23024b None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:02:13.440897 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-d4b101ec-a64f-434c-b81b-5caa9f23024b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:13.614422 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b101ec-a64f-434c-b81b-5caa9f23024b None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:02:13.692116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:13.800978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762338f-52d6-46c0-8b39-959e0fea3fc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:02:13.801590 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762338f-52d6-46c0-8b39-959e0fea3fc5 service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:13.801870 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762338f-52d6-46c0-8b39-959e0fea3fc5 service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:13.802134 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762338f-52d6-46c0-8b39-959e0fea3fc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:13.803838 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-2762338f-52d6-46c0-8b39-959e0fea3fc5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:13.902835 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762338f-52d6-46c0-8b39-959e0fea3fc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:02:13.904501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762338f-52d6-46c0-8b39-959e0fea3fc5 service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:02:13.928756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 716/1406] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:02:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:02:15.907104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6ccab-0623-40f6-924d-2c5b66fefe7c None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:15.907104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6ccab-0623-40f6-924d-2c5b66fefe7c None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:15.907104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6ccab-0623-40f6-924d-2c5b66fefe7c None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:15.908914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 691/1407] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:02:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:15.914359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41055f7c-5119-43e3-ab54-d7587c7069c8 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:15.914852 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41055f7c-5119-43e3-ab54-d7587c7069c8 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:15.915035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41055f7c-5119-43e3-ab54-d7587c7069c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:15.989955 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-41055f7c-5119-43e3-ab54-d7587c7069c8 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:15.991300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-41055f7c-5119-43e3-ab54-d7587c7069c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:15.993380 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:16.108469 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 717/1408] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:16.402791 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6ccab-0623-40f6-924d-2c5b66fefe7c None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:02:16.404423 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-aff6ccab-0623-40f6-924d-2c5b66fefe7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:16.570014 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6ccab-0623-40f6-924d-2c5b66fefe7c None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:02:16.654206 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:16.705771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5621d0-e5dd-46a7-b0b2-333f3968d81f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:02:16.706565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5621d0-e5dd-46a7-b0b2-333f3968d81f service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:16.706851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5621d0-e5dd-46a7-b0b2-333f3968d81f service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:16.707169 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5621d0-e5dd-46a7-b0b2-333f3968d81f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:16.708779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6d5621d0-e5dd-46a7-b0b2-333f3968d81f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:16.811218 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5621d0-e5dd-46a7-b0b2-333f3968d81f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:02:16.813672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5621d0-e5dd-46a7-b0b2-333f3968d81f service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:02:16.832522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 692/1409] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:02:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:02:26.106355 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=42 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:26.830120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=64 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:32.295237 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90737d08-f1da-4a77-8ed1-e46758954785 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:32.295774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90737d08-f1da-4a77-8ed1-e46758954785 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:32.295774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90737d08-f1da-4a77-8ed1-e46758954785 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:32.297034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 718/1410] 10.222.0.36 () {54 vars in 918 bytes} [Fri Sep 2 09:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:32.302038 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc36dff-0bfc-41eb-ab66-33954dc5f3d1 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:32.302500 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc36dff-0bfc-41eb-ab66-33954dc5f3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:32.302599 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc36dff-0bfc-41eb-ab66-33954dc5f3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:32.384182 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-ffc36dff-0bfc-41eb-ab66-33954dc5f3d1 None None] MFA Rules not processed for user `6a0f0fdbbd20492d8c35ec4b73404632`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:32.385390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ffc36dff-0bfc-41eb-ab66-33954dc5f3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:32.490123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 693/1411] 10.222.0.36 () {58 vars in 1015 bytes} [Fri Sep 2 09:02:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:02:32.504244 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92f17e-d42b-426c-9f88-09a98d8a8b77 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:32.504533 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92f17e-d42b-426c-9f88-09a98d8a8b77 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:32.504809 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92f17e-d42b-426c-9f88-09a98d8a8b77 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:32.506140 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 719/1412] 10.222.0.36 () {54 vars in 918 bytes} [Fri Sep 2 09:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:32.510783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bb9c0-cd23-4ccb-894b-deba6e3fc789 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:32.511092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bb9c0-cd23-4ccb-894b-deba6e3fc789 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:32.511370 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593bb9c0-cd23-4ccb-894b-deba6e3fc789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:32.583647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:32.586720 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-593bb9c0-cd23-4ccb-894b-deba6e3fc789 None None] MFA Rules not processed for user `cce502a83f5343558c2136c0193e7d85`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:32.588486 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-593bb9c0-cd23-4ccb-894b-deba6e3fc789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:32.704630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 694/1413] 10.222.0.36 () {58 vars in 1015 bytes} [Fri Sep 2 09:02:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:35.028727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2fd13-4670-41a2-bd06-0708785b2e1d None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:35.029240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2fd13-4670-41a2-bd06-0708785b2e1d None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:35.029240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2fd13-4670-41a2-bd06-0708785b2e1d None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:35.030595 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 720/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:35.035924 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af5057-67b6-4408-9aeb-978c6ddcb8a4 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:35.036335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af5057-67b6-4408-9aeb-978c6ddcb8a4 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:35.036481 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af5057-67b6-4408-9aeb-978c6ddcb8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:35.101809 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-e7af5057-67b6-4408-9aeb-978c6ddcb8a4 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:35.103015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-e7af5057-67b6-4408-9aeb-978c6ddcb8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:35.157148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:35.201167 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 695/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:35.494302 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2fd13-4670-41a2-bd06-0708785b2e1d None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:02:35.496013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-3ea2fd13-4670-41a2-bd06-0708785b2e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:35.638555 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2fd13-4670-41a2-bd06-0708785b2e1d None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:02:35.709370 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:35.749450 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eeb48e-dcf6-487c-ad68-5fee361922e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:02:35.750049 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eeb48e-dcf6-487c-ad68-5fee361922e7 service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:35.750349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eeb48e-dcf6-487c-ad68-5fee361922e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:35.750617 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eeb48e-dcf6-487c-ad68-5fee361922e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:35.752253 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-22eeb48e-dcf6-487c-ad68-5fee361922e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:35.839774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eeb48e-dcf6-487c-ad68-5fee361922e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:02:35.841342 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eeb48e-dcf6-487c-ad68-5fee361922e7 service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:02:35.858505 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 721/1416] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:02:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:02:36.733712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357546ce-90ab-41c5-b9f9-4af4e0937b0d None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:36.734252 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357546ce-90ab-41c5-b9f9-4af4e0937b0d None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:36.734252 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357546ce-90ab-41c5-b9f9-4af4e0937b0d None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:36.735565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 696/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:36.740343 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763e870-b534-443f-8867-7b41b3cc5c6e None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:36.740771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763e870-b534-443f-8867-7b41b3cc5c6e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:36.740959 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8763e870-b534-443f-8867-7b41b3cc5c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:36.828375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-8763e870-b534-443f-8867-7b41b3cc5c6e None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:36.829578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8763e870-b534-443f-8867-7b41b3cc5c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:36.856005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:36.923898 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 722/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:37.073239 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357546ce-90ab-41c5-b9f9-4af4e0937b0d None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:02:37.075317 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-357546ce-90ab-41c5-b9f9-4af4e0937b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:37.216061 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357546ce-90ab-41c5-b9f9-4af4e0937b0d None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:02:37.254986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:37.326697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef88ee-46c3-444d-8c2a-8befc37dcbf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:02:37.327250 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef88ee-46c3-444d-8c2a-8befc37dcbf6 service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:37.327530 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef88ee-46c3-444d-8c2a-8befc37dcbf6 service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:37.327792 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef88ee-46c3-444d-8c2a-8befc37dcbf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:37.329373 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-d0ef88ee-46c3-444d-8c2a-8befc37dcbf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:37.419759 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef88ee-46c3-444d-8c2a-8befc37dcbf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:02:37.421362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef88ee-46c3-444d-8c2a-8befc37dcbf6 service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:02:37.439730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 697/1419] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:02:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:02:38.319009 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cf93d-8882-4b59-b99e-799214b43cab None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:38.319571 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cf93d-8882-4b59-b99e-799214b43cab None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:38.319571 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cf93d-8882-4b59-b99e-799214b43cab None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:38.320862 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 723/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:38.326252 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee6bc4-6500-4d35-9990-807ba202fe99 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:38.326704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee6bc4-6500-4d35-9990-807ba202fe99 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:38.326810 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee6bc4-6500-4d35-9990-807ba202fe99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:38.379717 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:38.401604 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-c2ee6bc4-6500-4d35-9990-807ba202fe99 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:38.403437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c2ee6bc4-6500-4d35-9990-807ba202fe99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:38.497622 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 698/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:38.646711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043cf93d-8882-4b59-b99e-799214b43cab None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:02:38.648262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-043cf93d-8882-4b59-b99e-799214b43cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:38.792429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043cf93d-8882-4b59-b99e-799214b43cab None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:02:38.805053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:38.904922 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81533ab7-5d57-434f-b8d1-284f31fdadae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:02:38.905462 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81533ab7-5d57-434f-b8d1-284f31fdadae service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:38.905737 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81533ab7-5d57-434f-b8d1-284f31fdadae service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:38.906006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81533ab7-5d57-434f-b8d1-284f31fdadae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:38.907620 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-81533ab7-5d57-434f-b8d1-284f31fdadae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:38.995437 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81533ab7-5d57-434f-b8d1-284f31fdadae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:02:38.997080 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81533ab7-5d57-434f-b8d1-284f31fdadae service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:02:39.014886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 724/1422] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:02:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:02:39.874332 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153524d6-062a-4c72-81b8-e0a0e5cd6dc2 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:39.874861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153524d6-062a-4c72-81b8-e0a0e5cd6dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:39.874861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153524d6-062a-4c72-81b8-e0a0e5cd6dc2 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:39.876251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 699/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:39.881062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82e159-db14-429c-8992-88164de2eac1 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:39.881448 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82e159-db14-429c-8992-88164de2eac1 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:39.881612 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82e159-db14-429c-8992-88164de2eac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:39.901805 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:39.957870 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-2e82e159-db14-429c-8992-88164de2eac1 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:39.959062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-2e82e159-db14-429c-8992-88164de2eac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:40.052937 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 725/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:40.204816 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153524d6-062a-4c72-81b8-e0a0e5cd6dc2 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:02:40.206248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-153524d6-062a-4c72-81b8-e0a0e5cd6dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:40.314279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:40.350089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153524d6-062a-4c72-81b8-e0a0e5cd6dc2 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:02:40.460896 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb3bcac-ad53-4519-ace3-1f04513b18e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:02:40.461427 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3bcac-ad53-4519-ace3-1f04513b18e8 service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:40.461709 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3bcac-ad53-4519-ace3-1f04513b18e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:40.461977 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb3bcac-ad53-4519-ace3-1f04513b18e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:40.463586 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-5eb3bcac-ad53-4519-ace3-1f04513b18e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:40.540978 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:40.555672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb3bcac-ad53-4519-ace3-1f04513b18e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:02:40.557279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb3bcac-ad53-4519-ace3-1f04513b18e8 service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:02:40.575750 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 700/1425] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:02:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:02:41.428897 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf8aa4-f1d1-43b7-9ae6-e805623d2383 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:41.429416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf8aa4-f1d1-43b7-9ae6-e805623d2383 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:41.429513 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf8aa4-f1d1-43b7-9ae6-e805623d2383 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:41.430804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 726/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:02:41.435848 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c43a33-2d72-4a13-91ed-610db6de515f None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:41.436286 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c43a33-2d72-4a13-91ed-610db6de515f None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:41.436418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c43a33-2d72-4a13-91ed-610db6de515f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:41.510840 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-63c43a33-2d72-4a13-91ed-610db6de515f None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:41.512568 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-63c43a33-2d72-4a13-91ed-610db6de515f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:41.606606 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 701/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:41.759222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf8aa4-f1d1-43b7-9ae6-e805623d2383 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:02:41.760759 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-4acf8aa4-f1d1-43b7-9ae6-e805623d2383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:41.843314 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:41.904392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf8aa4-f1d1-43b7-9ae6-e805623d2383 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:02:42.014099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c747e7-4508-4baa-884e-4e06d804cb47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:02:42.014701 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c747e7-4508-4baa-884e-4e06d804cb47 service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:42.014980 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c747e7-4508-4baa-884e-4e06d804cb47 service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:42.015280 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c747e7-4508-4baa-884e-4e06d804cb47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:42.016862 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-65c747e7-4508-4baa-884e-4e06d804cb47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:42.064330 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:42.107116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c747e7-4508-4baa-884e-4e06d804cb47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:02:42.108720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c747e7-4508-4baa-884e-4e06d804cb47 service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:02:42.126281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 727/1428] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:02:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:02:42.923959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495aa813-1aa7-4e98-8bbb-c4b4ccf51ce2 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:42.924504 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495aa813-1aa7-4e98-8bbb-c4b4ccf51ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:42.924504 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495aa813-1aa7-4e98-8bbb-c4b4ccf51ce2 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:42.925844 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 702/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:42.930573 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd82729-fe2c-49d2-90b5-75e0cb52bbe5 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:42.931021 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd82729-fe2c-49d2-90b5-75e0cb52bbe5 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:42.931155 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd82729-fe2c-49d2-90b5-75e0cb52bbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:43.005705 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-2cd82729-fe2c-49d2-90b5-75e0cb52bbe5 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:43.006894 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-2cd82729-fe2c-49d2-90b5-75e0cb52bbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:43.100185 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 728/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:43.249299 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495aa813-1aa7-4e98-8bbb-c4b4ccf51ce2 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:02:43.250755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-495aa813-1aa7-4e98-8bbb-c4b4ccf51ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:43.301514 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:43.397756 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495aa813-1aa7-4e98-8bbb-c4b4ccf51ce2 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:02:43.508498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e77b5c-11ea-470c-b411-dd57d83f610d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:02:43.509029 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e77b5c-11ea-470c-b411-dd57d83f610d service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:43.509309 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e77b5c-11ea-470c-b411-dd57d83f610d service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:43.509578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e77b5c-11ea-470c-b411-dd57d83f610d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:43.511256 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-f9e77b5c-11ea-470c-b411-dd57d83f610d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:43.518041 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:43.598405 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e77b5c-11ea-470c-b411-dd57d83f610d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:02:43.600047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e77b5c-11ea-470c-b411-dd57d83f610d service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:02:43.617914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 703/1431] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:02:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:02:44.427428 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3c5da-0622-42da-af2c-3851fe79b55b None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:44.428104 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3c5da-0622-42da-af2c-3851fe79b55b None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:44.428104 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3c5da-0622-42da-af2c-3851fe79b55b None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:44.429325 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 729/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:44.434333 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155dfefc-1e30-4635-890e-ed32372b7b53 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:44.434799 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155dfefc-1e30-4635-890e-ed32372b7b53 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:44.434904 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155dfefc-1e30-4635-890e-ed32372b7b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:44.509774 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-155dfefc-1e30-4635-890e-ed32372b7b53 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:44.510982 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-155dfefc-1e30-4635-890e-ed32372b7b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:44.579330 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:44.608192 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 704/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:44.760590 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3c5da-0622-42da-af2c-3851fe79b55b None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:02:44.762664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c3c3c5da-0622-42da-af2c-3851fe79b55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:44.780215 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:44.913542 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3c5da-0622-42da-af2c-3851fe79b55b None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:02:45.001161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:45.027351 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013838a6-a77c-48fe-b06a-eb25449cacc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:02:45.028014 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013838a6-a77c-48fe-b06a-eb25449cacc5 service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:45.028300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013838a6-a77c-48fe-b06a-eb25449cacc5 service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:45.028575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013838a6-a77c-48fe-b06a-eb25449cacc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:45.030170 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-013838a6-a77c-48fe-b06a-eb25449cacc5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:45.121569 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013838a6-a77c-48fe-b06a-eb25449cacc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:02:45.123341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013838a6-a77c-48fe-b06a-eb25449cacc5 service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:02:45.140840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 730/1434] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:02:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:02:45.943230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316db23b-609f-42b3-bcf5-34c534f2548c None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:45.943802 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316db23b-609f-42b3-bcf5-34c534f2548c None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:45.943802 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316db23b-609f-42b3-bcf5-34c534f2548c None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:45.945100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 705/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:45.950598 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30696a0c-a2e0-445c-84b1-862e5258ee08 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:45.951015 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30696a0c-a2e0-445c-84b1-862e5258ee08 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:45.951271 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30696a0c-a2e0-445c-84b1-862e5258ee08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:46.024101 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-30696a0c-a2e0-445c-84b1-862e5258ee08 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:46.025272 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-30696a0c-a2e0-445c-84b1-862e5258ee08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:46.066152 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:46.118742 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 731/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:46.267284 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316db23b-609f-42b3-bcf5-34c534f2548c None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:02:46.268710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-316db23b-609f-42b3-bcf5-34c534f2548c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:46.413575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316db23b-609f-42b3-bcf5-34c534f2548c None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:02:46.466842 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:46.524463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a4e66-4f97-46d6-8eb1-e5f43cf87255 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:02:46.525001 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a4e66-4f97-46d6-8eb1-e5f43cf87255 service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:46.525307 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a4e66-4f97-46d6-8eb1-e5f43cf87255 service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:46.525577 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a4e66-4f97-46d6-8eb1-e5f43cf87255 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:46.527231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-991a4e66-4f97-46d6-8eb1-e5f43cf87255 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:46.613320 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991a4e66-4f97-46d6-8eb1-e5f43cf87255 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:02:46.614882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991a4e66-4f97-46d6-8eb1-e5f43cf87255 service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:02:46.633161 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 706/1437] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:02:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:02:47.450981 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176a1d0-6255-4e13-94fd-23c9586493e7 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:47.451574 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176a1d0-6255-4e13-94fd-23c9586493e7 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:47.451574 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a176a1d0-6255-4e13-94fd-23c9586493e7 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:47.452890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 732/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:47.457725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6bff1-62a2-4285-99b9-12096aaf757c None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:47.458123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6bff1-62a2-4285-99b9-12096aaf757c None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:47.458274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6bff1-62a2-4285-99b9-12096aaf757c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:47.533453 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-b8a6bff1-62a2-4285-99b9-12096aaf757c None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:47.534569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b8a6bff1-62a2-4285-99b9-12096aaf757c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:47.536584 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:47.630870 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 707/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:47.780376 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176a1d0-6255-4e13-94fd-23c9586493e7 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:02:47.782329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a176a1d0-6255-4e13-94fd-23c9586493e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:47.923282 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a176a1d0-6255-4e13-94fd-23c9586493e7 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:02:47.950314 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:48.036204 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df82682f-3cfa-4301-9146-9e730bab857a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:02:48.036815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82682f-3cfa-4301-9146-9e730bab857a service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:48.037097 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82682f-3cfa-4301-9146-9e730bab857a service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:48.037372 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df82682f-3cfa-4301-9146-9e730bab857a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:48.038965 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-df82682f-3cfa-4301-9146-9e730bab857a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:48.128675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df82682f-3cfa-4301-9146-9e730bab857a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:02:48.130262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df82682f-3cfa-4301-9146-9e730bab857a service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:02:48.147656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 733/1440] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:02:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:02:49.947046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97ff90-c027-439d-bab2-6eb1559859f5 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:49.947594 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97ff90-c027-439d-bab2-6eb1559859f5 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:49.947594 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97ff90-c027-439d-bab2-6eb1559859f5 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:49.948878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 708/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:49.954139 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8aa32a-17e3-4086-a649-311fb0ae7bbc None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:49.954566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8aa32a-17e3-4086-a649-311fb0ae7bbc None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:49.954747 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8aa32a-17e3-4086-a649-311fb0ae7bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:49.998837 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:50.024324 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-1c8aa32a-17e3-4086-a649-311fb0ae7bbc None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:50.025537 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-1c8aa32a-17e3-4086-a649-311fb0ae7bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:50.124953 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 734/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:02:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:50.414772 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97ff90-c027-439d-bab2-6eb1559859f5 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:02:50.416377 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1d97ff90-c027-439d-bab2-6eb1559859f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:50.517563 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97ff90-c027-439d-bab2-6eb1559859f5 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:02:50.548821 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:50.589685 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40a7c9-98d3-4fad-8b59-3fc45c67020a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:02:50.590284 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40a7c9-98d3-4fad-8b59-3fc45c67020a service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:50.590561 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40a7c9-98d3-4fad-8b59-3fc45c67020a service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:50.590830 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40a7c9-98d3-4fad-8b59-3fc45c67020a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:50.592448 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-0e40a7c9-98d3-4fad-8b59-3fc45c67020a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:50.709163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40a7c9-98d3-4fad-8b59-3fc45c67020a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:02:50.710590 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40a7c9-98d3-4fad-8b59-3fc45c67020a service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:02:50.728189 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 709/1443] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:02:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:02:52.815059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a90fb-dd84-4149-a6a8-54f41ecd071e None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:52.815756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a90fb-dd84-4149-a6a8-54f41ecd071e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:52.815756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a90fb-dd84-4149-a6a8-54f41ecd071e None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:52.816994 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 735/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:52.822382 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8058a27-ec33-4fcf-bb42-6dec52a78e9c None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:52.822840 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8058a27-ec33-4fcf-bb42-6dec52a78e9c None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:52.822945 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8058a27-ec33-4fcf-bb42-6dec52a78e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:52.888628 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-b8058a27-ec33-4fcf-bb42-6dec52a78e9c None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:52.889816 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b8058a27-ec33-4fcf-bb42-6dec52a78e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:52.902122 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:52.984119 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 710/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:02:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:53.277445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a90fb-dd84-4149-a6a8-54f41ecd071e None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:02:53.278969 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-891a90fb-dd84-4149-a6a8-54f41ecd071e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:53.381107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a90fb-dd84-4149-a6a8-54f41ecd071e None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:02:53.393468 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:53.454028 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a03feb-b704-41b6-b21b-888627e0a652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:02:53.454622 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a03feb-b704-41b6-b21b-888627e0a652 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:53.454896 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a03feb-b704-41b6-b21b-888627e0a652 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:53.455199 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a03feb-b704-41b6-b21b-888627e0a652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:53.456775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-46a03feb-b704-41b6-b21b-888627e0a652 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:53.561941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a03feb-b704-41b6-b21b-888627e0a652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:02:53.563575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a03feb-b704-41b6-b21b-888627e0a652 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:02:53.581423 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 736/1446] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:02:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:02:55.844598 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283da913-b6d3-4fb0-bafd-141ba3e20398 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:55.845128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283da913-b6d3-4fb0-bafd-141ba3e20398 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:55.845128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283da913-b6d3-4fb0-bafd-141ba3e20398 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:55.846443 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 711/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:55.851729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13c2eb-2097-465e-868d-cbd1a06d6b89 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:55.852136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13c2eb-2097-465e-868d-cbd1a06d6b89 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:55.852279 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13c2eb-2097-465e-868d-cbd1a06d6b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:55.900208 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:55.918716 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-aa13c2eb-2097-465e-868d-cbd1a06d6b89 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:55.919920 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-aa13c2eb-2097-465e-868d-cbd1a06d6b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:56.013064 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 737/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:56.295711 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283da913-b6d3-4fb0-bafd-141ba3e20398 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:02:56.297288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-283da913-b6d3-4fb0-bafd-141ba3e20398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:56.401635 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283da913-b6d3-4fb0-bafd-141ba3e20398 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:02:56.441750 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:56.474660 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f6540-125c-4caa-a597-545d3219754d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:02:56.475302 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f6540-125c-4caa-a597-545d3219754d service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:56.475581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f6540-125c-4caa-a597-545d3219754d service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:56.475853 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f6540-125c-4caa-a597-545d3219754d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:56.477450 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-530f6540-125c-4caa-a597-545d3219754d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:56.583713 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f6540-125c-4caa-a597-545d3219754d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:02:56.585176 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530f6540-125c-4caa-a597-545d3219754d service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:02:56.603274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 712/1449] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:02:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:02:58.628915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7db3f0-1f7c-4b86-82b1-30733b9bfb1e None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:58.629474 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7db3f0-1f7c-4b86-82b1-30733b9bfb1e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:58.629474 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7db3f0-1f7c-4b86-82b1-30733b9bfb1e None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:58.630756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 738/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:02:58.636063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347ee73-e960-498b-8646-a82e64975294 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:58.636983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347ee73-e960-498b-8646-a82e64975294 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:58.637104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4347ee73-e960-498b-8646-a82e64975294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:58.702598 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-4347ee73-e960-498b-8646-a82e64975294 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:02:58.703829 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-4347ee73-e960-498b-8646-a82e64975294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:58.723792 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:58.798445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 713/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:02:59.083094 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7db3f0-1f7c-4b86-82b1-30733b9bfb1e None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:02:59.084669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-7c7db3f0-1f7c-4b86-82b1-30733b9bfb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:59.187731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7db3f0-1f7c-4b86-82b1-30733b9bfb1e None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:02:59.207653 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:02:59.260407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150d0f5-0930-4716-bb38-0ae258a66f1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:02:59.260998 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150d0f5-0930-4716-bb38-0ae258a66f1a service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:02:59.261273 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150d0f5-0930-4716-bb38-0ae258a66f1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:02:59.261541 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150d0f5-0930-4716-bb38-0ae258a66f1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:02:59.263149 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a150d0f5-0930-4716-bb38-0ae258a66f1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:02:59.381624 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150d0f5-0930-4716-bb38-0ae258a66f1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:02:59.383092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150d0f5-0930-4716-bb38-0ae258a66f1a service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:02:59.402759 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 739/1452] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:02:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:03:01.370236 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe400c14-be6c-40bb-9f8b-56890ab23af0 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:01.370680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe400c14-be6c-40bb-9f8b-56890ab23af0 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:01.371352 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe400c14-be6c-40bb-9f8b-56890ab23af0 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:01.372647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 714/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:03:01.378070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239b97b-0e31-4906-a39e-db24396ba625 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:01.378456 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239b97b-0e31-4906-a39e-db24396ba625 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:01.378623 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239b97b-0e31-4906-a39e-db24396ba625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:01.441610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:01.444547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-a239b97b-0e31-4906-a39e-db24396ba625 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:03:01.446247 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a239b97b-0e31-4906-a39e-db24396ba625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:01.542486 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 740/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:03:01.833002 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe400c14-be6c-40bb-9f8b-56890ab23af0 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:03:01.834545 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-fe400c14-be6c-40bb-9f8b-56890ab23af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:01.935680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe400c14-be6c-40bb-9f8b-56890ab23af0 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:03:01.984639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:02.008925 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce06128e-f019-4d05-9db9-0af072eff7f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:03:02.009544 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06128e-f019-4d05-9db9-0af072eff7f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:02.009814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06128e-f019-4d05-9db9-0af072eff7f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:02.010074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce06128e-f019-4d05-9db9-0af072eff7f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:02.011659 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ce06128e-f019-4d05-9db9-0af072eff7f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:02.121314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06128e-f019-4d05-9db9-0af072eff7f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:03:02.122767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce06128e-f019-4d05-9db9-0af072eff7f5 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:03:02.140524 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 715/1455] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:03:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:03:04.101561 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31437d-73fd-46ba-a81c-6b00ffe16855 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:04.102075 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31437d-73fd-46ba-a81c-6b00ffe16855 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:04.102193 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31437d-73fd-46ba-a81c-6b00ffe16855 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:04.103485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 741/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:03:04.108648 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590cad2-446e-4a74-9740-43818693912d None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:04.109069 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590cad2-446e-4a74-9740-43818693912d None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:04.109208 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590cad2-446e-4a74-9740-43818693912d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:04.174730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-b590cad2-446e-4a74-9740-43818693912d None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:03:04.176032 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b590cad2-446e-4a74-9740-43818693912d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:04.205894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:04.273397 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 716/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:03:04.553229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31437d-73fd-46ba-a81c-6b00ffe16855 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:03:04.554771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0d31437d-73fd-46ba-a81c-6b00ffe16855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:04.656004 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31437d-73fd-46ba-a81c-6b00ffe16855 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:03:04.684432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:04.732304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384fecad-6057-4589-aab4-88ab5d00ac44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:03:04.732906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fecad-6057-4589-aab4-88ab5d00ac44 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:04.733184 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fecad-6057-4589-aab4-88ab5d00ac44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:04.733454 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fecad-6057-4589-aab4-88ab5d00ac44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:04.735030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-384fecad-6057-4589-aab4-88ab5d00ac44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:04.842333 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384fecad-6057-4589-aab4-88ab5d00ac44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:03:04.843788 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384fecad-6057-4589-aab4-88ab5d00ac44 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:03:04.861392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 742/1458] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:03:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:03:06.815451 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021da04-a8e9-4284-8248-677e3feae6c7 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:06.816010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021da04-a8e9-4284-8248-677e3feae6c7 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:06.816010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7021da04-a8e9-4284-8248-677e3feae6c7 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:06.817352 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 717/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:03:06.822653 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b6930-4b64-4298-9341-1f75fdab8f4b None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:06.823116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b6930-4b64-4298-9341-1f75fdab8f4b None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:06.823267 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b6930-4b64-4298-9341-1f75fdab8f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:06.888359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-3d9b6930-4b64-4298-9341-1f75fdab8f4b None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:03:06.889476 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-3d9b6930-4b64-4298-9341-1f75fdab8f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:06.904378 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:06.999252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 743/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:03:07.278066 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021da04-a8e9-4284-8248-677e3feae6c7 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:03:07.279633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7021da04-a8e9-4284-8248-677e3feae6c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:07.380879 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7021da04-a8e9-4284-8248-677e3feae6c7 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:03:07.434972 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:07.454213 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0c53d-bea9-44ac-88f6-eac9d00a1ac0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:03:07.454822 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0c53d-bea9-44ac-88f6-eac9d00a1ac0 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:07.455132 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0c53d-bea9-44ac-88f6-eac9d00a1ac0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:07.455406 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0c53d-bea9-44ac-88f6-eac9d00a1ac0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:07.457011 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-03a0c53d-bea9-44ac-88f6-eac9d00a1ac0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:07.564238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a0c53d-bea9-44ac-88f6-eac9d00a1ac0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:03:07.565585 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a0c53d-bea9-44ac-88f6-eac9d00a1ac0 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:03:07.583690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 718/1461] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:03:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:03:09.514389 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48accb1-5325-4c44-8304-1ea618607ee0 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:09.514960 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48accb1-5325-4c44-8304-1ea618607ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:09.514960 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48accb1-5325-4c44-8304-1ea618607ee0 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:09.516341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 744/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:03:09.521389 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffb0b8-29d4-454a-9746-93dc8d8c98d4 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:09.521822 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffb0b8-29d4-454a-9746-93dc8d8c98d4 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:09.521924 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffb0b8-29d4-454a-9746-93dc8d8c98d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:09.587081 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-deffb0b8-29d4-454a-9746-93dc8d8c98d4 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:03:09.588293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-deffb0b8-29d4-454a-9746-93dc8d8c98d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:09.626384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:09.684328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 719/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:03:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:03:09.969636 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48accb1-5325-4c44-8304-1ea618607ee0 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:03:09.971275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-b48accb1-5325-4c44-8304-1ea618607ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:10.075004 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48accb1-5325-4c44-8304-1ea618607ee0 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:03:10.111514 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:10.146873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bad409-c3b4-47d7-830e-1f92be99e372 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:03:10.147502 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bad409-c3b4-47d7-830e-1f92be99e372 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:10.147785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bad409-c3b4-47d7-830e-1f92be99e372 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:10.148049 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bad409-c3b4-47d7-830e-1f92be99e372 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:10.149607 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-91bad409-c3b4-47d7-830e-1f92be99e372 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:10.256375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bad409-c3b4-47d7-830e-1f92be99e372 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:03:10.257750 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bad409-c3b4-47d7-830e-1f92be99e372 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:03:10.275300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 745/1464] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:03:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:03:12.217548 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c3ee5-df98-4a6c-854b-0c6cc5ab83e3 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:12.218113 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c3ee5-df98-4a6c-854b-0c6cc5ab83e3 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:12.218113 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c3ee5-df98-4a6c-854b-0c6cc5ab83e3 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:12.219471 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 720/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:03:12.224681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952c814-29e3-45c5-9599-f91d92200063 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:12.225142 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952c814-29e3-45c5-9599-f91d92200063 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:12.225247 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952c814-29e3-45c5-9599-f91d92200063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:12.290631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-a952c814-29e3-45c5-9599-f91d92200063 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:03:12.291849 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a952c814-29e3-45c5-9599-f91d92200063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:12.310164 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:12.389352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 746/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:03:12.673599 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c3ee5-df98-4a6c-854b-0c6cc5ab83e3 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:03:12.675161 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7b1c3ee5-df98-4a6c-854b-0c6cc5ab83e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:12.774867 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c3ee5-df98-4a6c-854b-0c6cc5ab83e3 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:03:12.840500 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:12.847890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92951a-4cdf-40e2-8912-5b0321ff0936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:03:12.848515 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92951a-4cdf-40e2-8912-5b0321ff0936 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:12.848802 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92951a-4cdf-40e2-8912-5b0321ff0936 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:12.849081 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92951a-4cdf-40e2-8912-5b0321ff0936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:12.850685 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-bf92951a-4cdf-40e2-8912-5b0321ff0936 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:12.962188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92951a-4cdf-40e2-8912-5b0321ff0936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:03:12.963546 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92951a-4cdf-40e2-8912-5b0321ff0936 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:03:12.981702 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 721/1467] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:03:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:03:14.895085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb96474-01c3-4d99-b562-7054e225ed67 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:14.895631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb96474-01c3-4d99-b562-7054e225ed67 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:14.895752 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb96474-01c3-4d99-b562-7054e225ed67 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:14.897353 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 747/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:03:14.903892 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baec0d2-7293-406d-9401-3fce83ba927f None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:14.904325 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baec0d2-7293-406d-9401-3fce83ba927f None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:14.904447 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baec0d2-7293-406d-9401-3fce83ba927f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:14.972031 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-1baec0d2-7293-406d-9401-3fce83ba927f None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:03:14.973250 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1baec0d2-7293-406d-9401-3fce83ba927f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:15.020709 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:15.069672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 722/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:03:15.353183 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb96474-01c3-4d99-b562-7054e225ed67 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:03:15.354701 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-cdb96474-01c3-4d99-b562-7054e225ed67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:15.456757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb96474-01c3-4d99-b562-7054e225ed67 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:03:15.505236 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:15.531600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c0abf-166e-473a-9b5c-fbca2756deec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:03:15.532202 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c0abf-166e-473a-9b5c-fbca2756deec service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:15.532480 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c0abf-166e-473a-9b5c-fbca2756deec service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:15.532749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c0abf-166e-473a-9b5c-fbca2756deec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:15.534346 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-128c0abf-166e-473a-9b5c-fbca2756deec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:15.643303 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c0abf-166e-473a-9b5c-fbca2756deec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:03:15.644741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c0abf-166e-473a-9b5c-fbca2756deec service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:03:15.662589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 748/1470] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:03:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:03:25.067851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=22 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:25.660474 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=66 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:35.091361 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f80f68-5165-437e-a9d7-a2fef741f9fa None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:35.091940 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f80f68-5165-437e-a9d7-a2fef741f9fa None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:35.091940 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f80f68-5165-437e-a9d7-a2fef741f9fa None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:35.093252 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 723/1471] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:03:35.098291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d3b6b-6bfa-4aec-b646-34f18eca9e4b None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:35.098687 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d3b6b-6bfa-4aec-b646-34f18eca9e4b None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:35.098846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d3b6b-6bfa-4aec-b646-34f18eca9e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:35.163857 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-512d3b6b-6bfa-4aec-b646-34f18eca9e4b None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:03:35.165065 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-512d3b6b-6bfa-4aec-b646-34f18eca9e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:35.262181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 749/1472] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:03:35.557339 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d7b88-bd05-40c2-9e56-98a79bf2eb1d None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:35.557625 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d7b88-bd05-40c2-9e56-98a79bf2eb1d None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:35.558409 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d7b88-bd05-40c2-9e56-98a79bf2eb1d None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:35.559781 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 724/1473] 10.222.0.36 () {54 vars in 961 bytes} [Fri Sep 2 09:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:03:35.564990 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466c20e-d6a0-46fb-bdb5-c2f635cf1388 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:35.565397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466c20e-d6a0-46fb-bdb5-c2f635cf1388 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:35.565680 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466c20e-d6a0-46fb-bdb5-c2f635cf1388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:35.642038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-f466c20e-d6a0-46fb-bdb5-c2f635cf1388 None None] MFA Rules not processed for user `2474d89f10764afbbc5e740dccaae4df`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:03:35.643170 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f466c20e-d6a0-46fb-bdb5-c2f635cf1388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:35.663005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:35.718189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 750/1474] 10.222.0.36 () {58 vars in 1058 bytes} [Fri Sep 2 09:03:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:03:35.725590 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d7b88-bd05-40c2-9e56-98a79bf2eb1d None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:03:35.727187 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-2d5d7b88-bd05-40c2-9e56-98a79bf2eb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:35.828906 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d7b88-bd05-40c2-9e56-98a79bf2eb1d None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:03:35.900180 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c2aa5-09da-4a84-85b6-1e43f0ba27d3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2474d89f10764afbbc5e740dccaae4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:03:35.900712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c2aa5-09da-4a84-85b6-1e43f0ba27d3 service glance] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:35.901014 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c2aa5-09da-4a84-85b6-1e43f0ba27d3 service glance] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:35.901282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c2aa5-09da-4a84-85b6-1e43f0ba27d3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:35.902846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1f6c2aa5-09da-4a84-85b6-1e43f0ba27d3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:35.955597 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:36.010165 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6c2aa5-09da-4a84-85b6-1e43f0ba27d3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:03:36.011639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6c2aa5-09da-4a84-85b6-1e43f0ba27d3 service glance] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:03:36.029552 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 725/1475] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:03:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:03:38.004807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3416d-8165-4f6b-9cd7-9f0e95a65a71 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:38.005386 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3416d-8165-4f6b-9cd7-9f0e95a65a71 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:38.005386 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f3416d-8165-4f6b-9cd7-9f0e95a65a71 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:38.006698 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 751/1476] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:03:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:03:38.011925 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd4af8-7141-4255-ab87-1c0b50cf2af2 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:38.012354 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd4af8-7141-4255-ab87-1c0b50cf2af2 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:38.012553 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd4af8-7141-4255-ab87-1c0b50cf2af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:38.078711 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-bedd4af8-7141-4255-ab87-1c0b50cf2af2 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:03:38.079960 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-bedd4af8-7141-4255-ab87-1c0b50cf2af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:38.173709 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 726/1477] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:03:38.443592 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3416d-8165-4f6b-9cd7-9f0e95a65a71 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:03:38.445116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-33f3416d-8165-4f6b-9cd7-9f0e95a65a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:38.568955 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f3416d-8165-4f6b-9cd7-9f0e95a65a71 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:03:38.616071 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:38.660626 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08f7f7-0197-488d-a497-fd48895345f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:03:38.661226 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08f7f7-0197-488d-a497-fd48895345f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:38.661503 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08f7f7-0197-488d-a497-fd48895345f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:38.661771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08f7f7-0197-488d-a497-fd48895345f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:38.662399 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc08f7f7-0197-488d-a497-fd48895345f4 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:03:38.664212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: WARNING py.warnings [None req-bc08f7f7-0197-488d-a497-fd48895345f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:03:38.664212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: warnings.warn(msg) Sep 02 09:03:38.664212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: Sep 02 09:03:38.665359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc08f7f7-0197-488d-a497-fd48895345f4 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:03:38.684292 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 752/1478] 10.222.0.36 () {56 vars in 1108 bytes} [Fri Sep 2 09:03:38 2022] GET /identity/v3/projects => generated 2553 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:03:40.594152 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db512231-ce1a-4cde-b597-4fe8307042d2 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:40.594686 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db512231-ce1a-4cde-b597-4fe8307042d2 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:40.594686 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db512231-ce1a-4cde-b597-4fe8307042d2 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:40.596066 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 727/1479] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:03:40.601136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5640cf-c4fc-4c0e-8925-464a3acc0c45 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:40.601552 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5640cf-c4fc-4c0e-8925-464a3acc0c45 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:40.601699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5640cf-c4fc-4c0e-8925-464a3acc0c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:40.668171 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-8e5640cf-c4fc-4c0e-8925-464a3acc0c45 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:03:40.669299 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8e5640cf-c4fc-4c0e-8925-464a3acc0c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:40.763557 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 753/1480] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:03:41.060654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db512231-ce1a-4cde-b597-4fe8307042d2 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:03:41.062237 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-db512231-ce1a-4cde-b597-4fe8307042d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:41.072388 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:41.204630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db512231-ce1a-4cde-b597-4fe8307042d2 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:03:41.281745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:41.315406 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785daf0c-36bc-4853-a38e-7cec44ca2d48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:03:41.315950 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785daf0c-36bc-4853-a38e-7cec44ca2d48 service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:41.316231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785daf0c-36bc-4853-a38e-7cec44ca2d48 service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:41.316505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785daf0c-36bc-4853-a38e-7cec44ca2d48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:41.318113 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-785daf0c-36bc-4853-a38e-7cec44ca2d48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:41.407162 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785daf0c-36bc-4853-a38e-7cec44ca2d48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:03:41.408755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785daf0c-36bc-4853-a38e-7cec44ca2d48 service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:03:41.426946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 728/1481] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:03:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:03:43.500370 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e72e3-c595-4d37-8e0b-ccabab469c98 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:43.500924 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e72e3-c595-4d37-8e0b-ccabab469c98 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:43.500924 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e72e3-c595-4d37-8e0b-ccabab469c98 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:43.502247 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 754/1482] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:03:43.507391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be6066-bbfb-46fd-b0c2-5acf4510b886 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:43.507824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be6066-bbfb-46fd-b0c2-5acf4510b886 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:43.507980 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be6066-bbfb-46fd-b0c2-5acf4510b886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:43.573340 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-36be6066-bbfb-46fd-b0c2-5acf4510b886 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:03:43.574562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-36be6066-bbfb-46fd-b0c2-5acf4510b886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:43.620402 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:43.669703 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 729/1483] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:03:43.953305 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e72e3-c595-4d37-8e0b-ccabab469c98 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:03:43.954857 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-4e3e72e3-c595-4d37-8e0b-ccabab469c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:43.977945 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:44.062234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e72e3-c595-4d37-8e0b-ccabab469c98 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:03:44.135030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d3549-0144-4cd8-8c1c-235a06a837bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:03:44.135659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d3549-0144-4cd8-8c1c-235a06a837bc service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:44.135959 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d3549-0144-4cd8-8c1c-235a06a837bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:44.136233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d3549-0144-4cd8-8c1c-235a06a837bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:44.137851 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-de5d3549-0144-4cd8-8c1c-235a06a837bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:44.341440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:44.377930 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d3549-0144-4cd8-8c1c-235a06a837bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:03:44.379421 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5d3549-0144-4cd8-8c1c-235a06a837bc service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:03:44.397648 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 755/1484] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:03:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:03:46.415129 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf900c-96f4-4f5c-8236-2b550a3144e8 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:46.415708 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf900c-96f4-4f5c-8236-2b550a3144e8 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:46.415708 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf900c-96f4-4f5c-8236-2b550a3144e8 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:46.417043 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 730/1485] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:03:46.423054 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ccbb8-af0c-4fa6-9731-c5f85749ce7a None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:46.423533 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ccbb8-af0c-4fa6-9731-c5f85749ce7a None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:46.423642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ccbb8-af0c-4fa6-9731-c5f85749ce7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:46.488416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-c06ccbb8-af0c-4fa6-9731-c5f85749ce7a None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:03:46.489613 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c06ccbb8-af0c-4fa6-9731-c5f85749ce7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:46.603839 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 756/1486] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:03:46.898855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf900c-96f4-4f5c-8236-2b550a3144e8 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:03:46.900442 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-74cf900c-96f4-4f5c-8236-2b550a3144e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:47.033980 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf900c-96f4-4f5c-8236-2b550a3144e8 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:03:47.105527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:47.107360 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f1ccd-6be5-4ed5-b5c1-56fa04af7da6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:03:47.107962 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f1ccd-6be5-4ed5-b5c1-56fa04af7da6 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:47.108251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f1ccd-6be5-4ed5-b5c1-56fa04af7da6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:47.108762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f1ccd-6be5-4ed5-b5c1-56fa04af7da6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:47.110689 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-d28f1ccd-6be5-4ed5-b5c1-56fa04af7da6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:47.217350 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f1ccd-6be5-4ed5-b5c1-56fa04af7da6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:03:47.218687 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f1ccd-6be5-4ed5-b5c1-56fa04af7da6 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:03:47.238610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 731/1487] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:03:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:03:49.277766 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb9b5b-7e87-4d7d-90a3-802cae6cdf1c None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:49.278346 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb9b5b-7e87-4d7d-90a3-802cae6cdf1c None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:49.278346 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cb9b5b-7e87-4d7d-90a3-802cae6cdf1c None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:49.279723 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 757/1488] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:03:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:03:49.285081 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887be89-726c-4961-ba92-7cb27aafdfec None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:49.285507 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887be89-726c-4961-ba92-7cb27aafdfec None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:49.285640 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7887be89-726c-4961-ba92-7cb27aafdfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:49.351876 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-7887be89-726c-4961-ba92-7cb27aafdfec None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:03:49.353084 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7887be89-726c-4961-ba92-7cb27aafdfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:49.406190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:49.450793 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 732/1489] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:03:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:03:49.740708 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb9b5b-7e87-4d7d-90a3-802cae6cdf1c None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:03:49.742224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-75cb9b5b-7e87-4d7d-90a3-802cae6cdf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:49.778066 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:49.845946 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cb9b5b-7e87-4d7d-90a3-802cae6cdf1c None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:03:49.918403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb062b-8dc8-491a-a1f9-163c9057daa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:03:49.918991 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb062b-8dc8-491a-a1f9-163c9057daa1 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:49.919291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb062b-8dc8-491a-a1f9-163c9057daa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:49.919557 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb062b-8dc8-491a-a1f9-163c9057daa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:49.921119 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-d8cb062b-8dc8-491a-a1f9-163c9057daa1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:49.999303 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:50.028170 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb062b-8dc8-491a-a1f9-163c9057daa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:03:50.029593 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb062b-8dc8-491a-a1f9-163c9057daa1 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:03:50.047469 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 758/1490] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:03:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:03:52.147245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7015a6e-20ee-42e2-b533-fbebb706c1d8 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:52.147764 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7015a6e-20ee-42e2-b533-fbebb706c1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:52.147764 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7015a6e-20ee-42e2-b533-fbebb706c1d8 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:52.149127 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 733/1491] 10.222.0.36 () {54 vars in 927 bytes} [Fri Sep 2 09:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:03:52.154270 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdbf29-2d13-46c0-b7fe-1439f2d8df15 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:52.154707 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdbf29-2d13-46c0-b7fe-1439f2d8df15 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:52.154819 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdbf29-2d13-46c0-b7fe-1439f2d8df15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:52.219443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-f6bdbf29-2d13-46c0-b7fe-1439f2d8df15 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:03:52.220624 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f6bdbf29-2d13-46c0-b7fe-1439f2d8df15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:52.317035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 759/1492] 10.222.0.36 () {58 vars in 1024 bytes} [Fri Sep 2 09:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:03:52.603801 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7015a6e-20ee-42e2-b533-fbebb706c1d8 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:03:52.605368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-e7015a6e-20ee-42e2-b533-fbebb706c1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:52.829446 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7015a6e-20ee-42e2-b533-fbebb706c1d8 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:03:52.900316 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c408f066-9fcc-4161-b8a1-9fbe4438674d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:03:52.900912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408f066-9fcc-4161-b8a1-9fbe4438674d service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:03:52.901187 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408f066-9fcc-4161-b8a1-9fbe4438674d service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:03:52.901476 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c408f066-9fcc-4161-b8a1-9fbe4438674d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:03:52.903103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c408f066-9fcc-4161-b8a1-9fbe4438674d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:03:52.911325 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:03:53.008359 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408f066-9fcc-4161-b8a1-9fbe4438674d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:03:53.009767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c408f066-9fcc-4161-b8a1-9fbe4438674d service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:03:53.027667 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 734/1493] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:04:02.315295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=82 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:04:03.025442 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=50 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:05:04.549659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10251b3-e35e-4227-9f64-a0a555532233 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:05:04.550223 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10251b3-e35e-4227-9f64-a0a555532233 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:05:04.550223 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10251b3-e35e-4227-9f64-a0a555532233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:05:04.625359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-f10251b3-e35e-4227-9f64-a0a555532233 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:05:04.626586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f10251b3-e35e-4227-9f64-a0a555532233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:05:04.722620 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 760/1494] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:05:04.730573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ad3c3-7b1c-4856-add1-a91300a17812 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:05:04.731091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ad3c3-7b1c-4856-add1-a91300a17812 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:05:04.731216 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ad3c3-7b1c-4856-add1-a91300a17812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:05:04.804410 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-284ad3c3-7b1c-4856-add1-a91300a17812 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:05:04.805550 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-284ad3c3-7b1c-4856-add1-a91300a17812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:05:04.900537 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 735/1495] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:05:04.905219 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10251b3-e35e-4227-9f64-a0a555532233 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:05:04.906415 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f10251b3-e35e-4227-9f64-a0a555532233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:05:04.917820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10251b3-e35e-4227-9f64-a0a555532233 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:05:04.978356 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:05:05.012700 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed06606-60a3-4882-a6ed-fb873b407977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:05:05.013291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed06606-60a3-4882-a6ed-fb873b407977 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:05:05.013566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed06606-60a3-4882-a6ed-fb873b407977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:05:05.013833 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed06606-60a3-4882-a6ed-fb873b407977 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:05:05.014463 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed06606-60a3-4882-a6ed-fb873b407977 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:05:05.014753 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed06606-60a3-4882-a6ed-fb873b407977 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:05:05.015782 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: WARNING py.warnings [None req-4ed06606-60a3-4882-a6ed-fb873b407977 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:05:05.015782 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: warnings.warn(msg) Sep 02 09:05:05.015782 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: Sep 02 09:05:05.016651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed06606-60a3-4882-a6ed-fb873b407977 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:05:05.027428 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 761/1496] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:05:05.035647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ad3c3-7b1c-4856-add1-a91300a17812 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:05:05.044797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ad3c3-7b1c-4856-add1-a91300a17812 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:05:05.106028 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:05:05.141160 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf6b26-a9e2-48c0-a3bb-78987811a854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:05:05.141765 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf6b26-a9e2-48c0-a3bb-78987811a854 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:05:05.142041 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf6b26-a9e2-48c0-a3bb-78987811a854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:05:05.142310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf6b26-a9e2-48c0-a3bb-78987811a854 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:05:05.142941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf6b26-a9e2-48c0-a3bb-78987811a854 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:05:05.144697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: WARNING py.warnings [None req-b0cf6b26-a9e2-48c0-a3bb-78987811a854 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:05:05.144697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: warnings.warn(msg) Sep 02 09:05:05.144697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: Sep 02 09:05:05.145890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf6b26-a9e2-48c0-a3bb-78987811a854 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:05:05.201712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 736/1497] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:05:05 2022] POST /identity/v3/projects => generated 375 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:05:05.205865 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed06606-60a3-4882-a6ed-fb873b407977 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:05:05.207378 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-4ed06606-60a3-4882-a6ed-fb873b407977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:05:05.239600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed06606-60a3-4882-a6ed-fb873b407977 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:05:05.330793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160bb044-69eb-4922-8a53-8f214ce2108c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:05:05.331414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160bb044-69eb-4922-8a53-8f214ce2108c admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:05:05.331692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160bb044-69eb-4922-8a53-8f214ce2108c admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:05:05.331960 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160bb044-69eb-4922-8a53-8f214ce2108c admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:05:05.332586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160bb044-69eb-4922-8a53-8f214ce2108c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:05:05.333589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: WARNING py.warnings [None req-160bb044-69eb-4922-8a53-8f214ce2108c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:05:05.333589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: warnings.warn(msg) Sep 02 09:05:05.333589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: Sep 02 09:05:05.334798 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160bb044-69eb-4922-8a53-8f214ce2108c admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:05:05.378631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 762/1498] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:05:05 2022] POST /identity/v3/users => generated 288 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:05:05.382771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf6b26-a9e2-48c0-a3bb-78987811a854 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:05:05.384543 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b0cf6b26-a9e2-48c0-a3bb-78987811a854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:05:05.445029 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf6b26-a9e2-48c0-a3bb-78987811a854 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:05:05.530446 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:05:05.541940 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319228cb-fe2c-4865-9f62-2937d3fb1117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:05:05.542546 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319228cb-fe2c-4865-9f62-2937d3fb1117 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:05:05.542826 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319228cb-fe2c-4865-9f62-2937d3fb1117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:05:05.543125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319228cb-fe2c-4865-9f62-2937d3fb1117 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:05:05.543771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319228cb-fe2c-4865-9f62-2937d3fb1117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:05:05.544778 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: WARNING py.warnings [None req-319228cb-fe2c-4865-9f62-2937d3fb1117 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:05:05.544778 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: warnings.warn(msg) Sep 02 09:05:05.544778 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: Sep 02 09:05:05.545661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319228cb-fe2c-4865-9f62-2937d3fb1117 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:05:05.558224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 737/1499] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:05:05.562454 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160bb044-69eb-4922-8a53-8f214ce2108c admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:05:05.564012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-160bb044-69eb-4922-8a53-8f214ce2108c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:05:05.576012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160bb044-69eb-4922-8a53-8f214ce2108c admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:05:05.600332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:05:05.670841 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4bc163-af51-45f4-8370-3029b7e02bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:05:05.671386 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bc163-af51-45f4-8370-3029b7e02bce admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:05:05.671659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bc163-af51-45f4-8370-3029b7e02bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:05:05.671924 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4bc163-af51-45f4-8370-3029b7e02bce admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:05:05.672551 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4bc163-af51-45f4-8370-3029b7e02bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:05:05.673549 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: WARNING py.warnings [None req-ec4bc163-af51-45f4-8370-3029b7e02bce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:05:05.673549 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: warnings.warn(msg) Sep 02 09:05:05.673549 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: Sep 02 09:05:05.674452 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4bc163-af51-45f4-8370-3029b7e02bce admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:05:05.686387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 763/1500] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:05:05.690551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319228cb-fe2c-4865-9f62-2937d3fb1117 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:05:05.692331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-319228cb-fe2c-4865-9f62-2937d3fb1117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:05:05.729809 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319228cb-fe2c-4865-9f62-2937d3fb1117 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:05:05.820937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68cfe66-22f0-4293-a645-744e35e42d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:05:05.821608 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cfe66-22f0-4293-a645-744e35e42d0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:05:05.821840 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cfe66-22f0-4293-a645-744e35e42d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:05:05.822110 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68cfe66-22f0-4293-a645-744e35e42d0a admin admin] PATH_INFO: `/v3/projects/fed7291fbcdc477d894b5b12e1461f4b/users/89d30c8a86d04826bbfdcd2a00777d97/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:05:05.843374 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68cfe66-22f0-4293-a645-744e35e42d0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fed7291fbcdc477d894b5b12e1461f4b, user_id=89d30c8a86d04826bbfdcd2a00777d97, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:05:05.844383 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: WARNING py.warnings [None req-b68cfe66-22f0-4293-a645-744e35e42d0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:05:05.844383 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: warnings.warn(msg) Sep 02 09:05:05.844383 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: Sep 02 09:05:05.846355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68cfe66-22f0-4293-a645-744e35e42d0a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:05:05.869870 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 738/1501] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:05:05 2022] PUT /identity/v3/projects/fed7291fbcdc477d894b5b12e1461f4b/users/89d30c8a86d04826bbfdcd2a00777d97/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:05:05.881733 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6e727-b50c-4f49-b0cb-58755b989e36 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:05:05.882097 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6e727-b50c-4f49-b0cb-58755b989e36 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:05:05.882508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e6e727-b50c-4f49-b0cb-58755b989e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:05:05.938590 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-62e6e727-b50c-4f49-b0cb-58755b989e36 None None] MFA Rules not processed for user `89d30c8a86d04826bbfdcd2a00777d97`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:05:05.939740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-62e6e727-b50c-4f49-b0cb-58755b989e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:05:06.016171 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 764/1502] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:05:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:05:06.029592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bcfc1-99de-4d3e-bd0e-f5ab18f6c98b None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:05:06.029880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bcfc1-99de-4d3e-bd0e-f5ab18f6c98b None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:05:06.030158 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61bcfc1-99de-4d3e-bd0e-f5ab18f6c98b None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:05:06.031553 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 739/1503] 10.222.0.36 () {54 vars in 964 bytes} [Fri Sep 2 09:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:05:06.036027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348f2f8-3cd7-4b39-b921-4eb5a0838834 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:05:06.036310 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348f2f8-3cd7-4b39-b921-4eb5a0838834 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:05:06.036580 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348f2f8-3cd7-4b39-b921-4eb5a0838834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:05:06.066917 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:05:06.112095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-f348f2f8-3cd7-4b39-b921-4eb5a0838834 None None] MFA Rules not processed for user `f8370135a78f414f9184c883ccb92767`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:05:06.113275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f348f2f8-3cd7-4b39-b921-4eb5a0838834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:05:06.188339 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 765/1504] 10.222.0.36 () {58 vars in 1061 bytes} [Fri Sep 2 09:05:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:05:06.198148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61bcfc1-99de-4d3e-bd0e-f5ab18f6c98b None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:05:06.199682 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-d61bcfc1-99de-4d3e-bd0e-f5ab18f6c98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:05:06.249080 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:05:06.303225 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61bcfc1-99de-4d3e-bd0e-f5ab18f6c98b None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:05:06.375608 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433131c7-7363-4aa8-8f88-cac51d4918b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:05:06.376212 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433131c7-7363-4aa8-8f88-cac51d4918b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:05:06.376488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433131c7-7363-4aa8-8f88-cac51d4918b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:05:06.376757 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433131c7-7363-4aa8-8f88-cac51d4918b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:05:06.378335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-433131c7-7363-4aa8-8f88-cac51d4918b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:05:06.474045 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433131c7-7363-4aa8-8f88-cac51d4918b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:05:06.475432 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433131c7-7363-4aa8-8f88-cac51d4918b0 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:05:06.478784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:05:06.494371 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 740/1505] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:05:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:05:06.538406 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5eb709-20b2-4eb1-8c84-2fe3c60b03c9 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:05:06.538691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5eb709-20b2-4eb1-8c84-2fe3c60b03c9 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:05:06.538964 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5eb709-20b2-4eb1-8c84-2fe3c60b03c9 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:05:06.540283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 766/1506] 10.222.0.36 () {52 vars in 830 bytes} [Fri Sep 2 09:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:05:06.544339 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433131c7-7363-4aa8-8f88-cac51d4918b0 service cinder] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:05:06.545539 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-433131c7-7363-4aa8-8f88-cac51d4918b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:05:06.640552 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433131c7-7363-4aa8-8f88-cac51d4918b0 service cinder] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:05:06.732145 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4735f2-46e8-4237-8949-df10945b65a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:05:06.732681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4735f2-46e8-4237-8949-df10945b65a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:05:06.732953 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4735f2-46e8-4237-8949-df10945b65a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:05:06.733218 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4735f2-46e8-4237-8949-df10945b65a1 admin admin] PATH_INFO: `/v3/users/89d30c8a86d04826bbfdcd2a00777d97` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:05:06.745735 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4735f2-46e8-4237-8949-df10945b65a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89d30c8a86d04826bbfdcd2a00777d97)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:05:06.746734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: WARNING py.warnings [None req-cc4735f2-46e8-4237-8949-df10945b65a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:05:06.746734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: warnings.warn(msg) Sep 02 09:05:06.746734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: Sep 02 09:05:06.747965 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4735f2-46e8-4237-8949-df10945b65a1 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:05:06.757226 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:05:06.813168 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-cc4735f2-46e8-4237-8949-df10945b65a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89d30c8a86d04826bbfdcd2a00777d97'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:05:06.813695 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-cc4735f2-46e8-4237-8949-df10945b65a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89d30c8a86d04826bbfdcd2a00777d97'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:05:06.837506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-cc4735f2-46e8-4237-8949-df10945b65a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89d30c8a86d04826bbfdcd2a00777d97'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:05:06.850872 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-cc4735f2-46e8-4237-8949-df10945b65a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89d30c8a86d04826bbfdcd2a00777d97'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:05:06.851341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-cc4735f2-46e8-4237-8949-df10945b65a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89d30c8a86d04826bbfdcd2a00777d97'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:05:06.865686 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 741/1507] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:05:06 2022] DELETE /identity/v3/users/89d30c8a86d04826bbfdcd2a00777d97 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:05:06.870093 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9241f-3ee5-419e-b54b-00cf8820d69e None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:05:06.870541 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9241f-3ee5-419e-b54b-00cf8820d69e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:05:06.870644 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9241f-3ee5-419e-b54b-00cf8820d69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:05:06.926963 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-e4d9241f-3ee5-419e-b54b-00cf8820d69e None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:05:06.928260 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-e4d9241f-3ee5-419e-b54b-00cf8820d69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:05:07.001653 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:05:07.015419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 767/1508] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:05:07.024792 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4735f2-46e8-4237-8949-df10945b65a1 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:05:07.026536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-cc4735f2-46e8-4237-8949-df10945b65a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:05:07.124772 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4735f2-46e8-4237-8949-df10945b65a1 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:05:07.178712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:05:07.199250 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea3597-28d0-4b11-821b-eb46fdc2f826 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:05:07.199859 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea3597-28d0-4b11-821b-eb46fdc2f826 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:05:07.200138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea3597-28d0-4b11-821b-eb46fdc2f826 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:05:07.200409 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea3597-28d0-4b11-821b-eb46fdc2f826 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:05:07.201985 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-2fea3597-28d0-4b11-821b-eb46fdc2f826 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:05:07.308934 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea3597-28d0-4b11-821b-eb46fdc2f826 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:05:07.310282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea3597-28d0-4b11-821b-eb46fdc2f826 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:05:07.328655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 742/1509] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:05:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:05:07.749216 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9241f-3ee5-419e-b54b-00cf8820d69e None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:05:07.750446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-e4d9241f-3ee5-419e-b54b-00cf8820d69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:05:07.763100 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9241f-3ee5-419e-b54b-00cf8820d69e None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:05:07.858178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbc08c-c8ad-473c-88ec-ee45b169d6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:05:07.858813 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbc08c-c8ad-473c-88ec-ee45b169d6bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:05:07.859116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbc08c-c8ad-473c-88ec-ee45b169d6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:05:07.859392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbc08c-c8ad-473c-88ec-ee45b169d6bd admin admin] PATH_INFO: `/v3/projects/fed7291fbcdc477d894b5b12e1461f4b` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:05:07.869019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fbc08c-c8ad-473c-88ec-ee45b169d6bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fed7291fbcdc477d894b5b12e1461f4b)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:05:07.870541 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: WARNING py.warnings [None req-41fbc08c-c8ad-473c-88ec-ee45b169d6bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:05:07.870541 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: warnings.warn(msg) Sep 02 09:05:07.870541 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: Sep 02 09:05:07.871772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fbc08c-c8ad-473c-88ec-ee45b169d6bd admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:05:07.927203 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-41fbc08c-c8ad-473c-88ec-ee45b169d6bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fed7291fbcdc477d894b5b12e1461f4b', 'request_id': 'req-41fbc08c-c8ad-473c-88ec-ee45b169d6bd'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:05:07.937414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-41fbc08c-c8ad-473c-88ec-ee45b169d6bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fed7291fbcdc477d894b5b12e1461f4b', 'request_id': 'req-41fbc08c-c8ad-473c-88ec-ee45b169d6bd'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:05:07.944595 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-41fbc08c-c8ad-473c-88ec-ee45b169d6bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fed7291fbcdc477d894b5b12e1461f4b', 'request_id': 'req-41fbc08c-c8ad-473c-88ec-ee45b169d6bd'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:05:07.974383 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-41fbc08c-c8ad-473c-88ec-ee45b169d6bd admin admin] The token cache is being invalidate because project fed7291fbcdc477d894b5b12e1461f4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:05:07.974667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-41fbc08c-c8ad-473c-88ec-ee45b169d6bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:05:07.975205 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-41fbc08c-c8ad-473c-88ec-ee45b169d6bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:05:07.976469 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 768/1510] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:05:07 2022] DELETE /identity/v3/projects/fed7291fbcdc477d894b5b12e1461f4b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:05:17.326289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=62 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:05:17.972534 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:06:36.107105 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9f0af-2c40-4517-932d-b6316be7921e None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:36.107716 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9f0af-2c40-4517-932d-b6316be7921e None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:36.107716 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9f0af-2c40-4517-932d-b6316be7921e None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:36.109533 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 743/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:06:36.117400 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ece29-1ce3-416e-bfdd-a224eef0da92 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:36.117867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ece29-1ce3-416e-bfdd-a224eef0da92 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:36.117974 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ece29-1ce3-416e-bfdd-a224eef0da92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:36.200138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-827ece29-1ce3-416e-bfdd-a224eef0da92 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:06:36.201345 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-827ece29-1ce3-416e-bfdd-a224eef0da92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:06:36.303156 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 769/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:06:36.457946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9f0af-2c40-4517-932d-b6316be7921e None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:06:36.460084 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-5af9f0af-2c40-4517-932d-b6316be7921e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:06:36.577331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9f0af-2c40-4517-932d-b6316be7921e None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:06:36.582746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623badf8-6231-43e9-8270-afbfb8808160 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:06:36.583381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623badf8-6231-43e9-8270-afbfb8808160 service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:36.583664 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623badf8-6231-43e9-8270-afbfb8808160 service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:36.583941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623badf8-6231-43e9-8270-afbfb8808160 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:36.591773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623badf8-6231-43e9-8270-afbfb8808160 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:06:36.593989 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623badf8-6231-43e9-8270-afbfb8808160 service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:06:36.603358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 744/1513] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:06:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:06:37.424380 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311e3dc-97f7-4019-8523-30d63a084d91 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:37.424910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311e3dc-97f7-4019-8523-30d63a084d91 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:37.424910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311e3dc-97f7-4019-8523-30d63a084d91 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:37.426245 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 770/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:06:37.431117 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6635c6f-24c8-43fa-905e-8e0a07bc3cda None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:37.431533 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6635c6f-24c8-43fa-905e-8e0a07bc3cda None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:37.431685 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6635c6f-24c8-43fa-905e-8e0a07bc3cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:37.452867 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-b6635c6f-24c8-43fa-905e-8e0a07bc3cda None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:06:37.454073 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b6635c6f-24c8-43fa-905e-8e0a07bc3cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:06:37.461913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 745/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:06:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:06:37.613359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311e3dc-97f7-4019-8523-30d63a084d91 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:06:37.628058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311e3dc-97f7-4019-8523-30d63a084d91 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:06:37.633420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964b2481-2916-4dc1-87cf-ae295c174190 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:06:37.634011 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b2481-2916-4dc1-87cf-ae295c174190 service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:37.634291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b2481-2916-4dc1-87cf-ae295c174190 service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:37.634564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964b2481-2916-4dc1-87cf-ae295c174190 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:37.642474 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964b2481-2916-4dc1-87cf-ae295c174190 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:06:37.644083 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964b2481-2916-4dc1-87cf-ae295c174190 service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:06:37.653396 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 771/1516] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:06:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:06:38.053359 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5910573-059c-44dd-8020-e3af09ead1d7 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:38.053563 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5910573-059c-44dd-8020-e3af09ead1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:38.053851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5910573-059c-44dd-8020-e3af09ead1d7 None None] PATH_INFO: `/v3` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:38.055162 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 746/1517] 10.222.0.18 () {54 vars in 925 bytes} [Fri Sep 2 09:06:38 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:06:38.063809 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87081922-4ad3-4e13-b15e-277ca6986dca None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:38.064091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87081922-4ad3-4e13-b15e-277ca6986dca None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:38.064363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87081922-4ad3-4e13-b15e-277ca6986dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:38.122376 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-87081922-4ad3-4e13-b15e-277ca6986dca None None] MFA Rules not processed for user `6a0f0fdbbd20492d8c35ec4b73404632`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:06:38.123600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-87081922-4ad3-4e13-b15e-277ca6986dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:06:38.139932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:06:38.195194 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 772/1518] 10.222.0.18 () {58 vars in 1015 bytes} [Fri Sep 2 09:06:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:06:47.048231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c3502-6d72-48e4-af3c-224f4ee9ae79 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:47.048803 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c3502-6d72-48e4-af3c-224f4ee9ae79 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:47.048803 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c3502-6d72-48e4-af3c-224f4ee9ae79 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:47.050106 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 747/1519] 10.222.0.36 () {54 vars in 961 bytes} [Fri Sep 2 09:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:06:47.054904 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed235f9-6df4-493e-b827-689e461ac6a0 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:47.055379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed235f9-6df4-493e-b827-689e461ac6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:47.055491 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed235f9-6df4-493e-b827-689e461ac6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:47.074915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-0ed235f9-6df4-493e-b827-689e461ac6a0 None None] MFA Rules not processed for user `6a0f0fdbbd20492d8c35ec4b73404632`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:06:47.076142 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0ed235f9-6df4-493e-b827-689e461ac6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:06:47.085043 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 773/1520] 10.222.0.36 () {58 vars in 1058 bytes} [Fri Sep 2 09:06:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:06:47.092279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c3502-6d72-48e4-af3c-224f4ee9ae79 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:06:47.106868 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c3502-6d72-48e4-af3c-224f4ee9ae79 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:06:47.111887 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0f585-e9e8-4e08-8434-06c8e81b36cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a0f0fdbbd20492d8c35ec4b73404632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:06:47.112409 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0f585-e9e8-4e08-8434-06c8e81b36cd service placement] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:47.112694 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0f585-e9e8-4e08-8434-06c8e81b36cd service placement] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:47.112963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0f585-e9e8-4e08-8434-06c8e81b36cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:47.121034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b0f585-e9e8-4e08-8434-06c8e81b36cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:06:47.122600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b0f585-e9e8-4e08-8434-06c8e81b36cd service placement] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:06:47.132280 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 748/1521] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:06:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:06:48.894769 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c2d37-8e44-44fd-961f-3c183490e8a9 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:48.895357 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c2d37-8e44-44fd-961f-3c183490e8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:48.895357 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c2d37-8e44-44fd-961f-3c183490e8a9 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:48.896695 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 774/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:06:48.901422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9185915-824d-4e59-86eb-3bf91a5b83c6 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:48.901860 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9185915-824d-4e59-86eb-3bf91a5b83c6 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:48.901967 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9185915-824d-4e59-86eb-3bf91a5b83c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:48.921223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-b9185915-824d-4e59-86eb-3bf91a5b83c6 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:06:48.922409 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b9185915-824d-4e59-86eb-3bf91a5b83c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:06:48.930407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 749/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:06:49.081622 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c2d37-8e44-44fd-961f-3c183490e8a9 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:06:49.095800 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c2d37-8e44-44fd-961f-3c183490e8a9 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:06:49.101312 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72675b-da4d-44ee-a38d-bdec77cee91c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:06:49.101922 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72675b-da4d-44ee-a38d-bdec77cee91c service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:49.102183 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72675b-da4d-44ee-a38d-bdec77cee91c service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:49.102479 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72675b-da4d-44ee-a38d-bdec77cee91c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:49.110495 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72675b-da4d-44ee-a38d-bdec77cee91c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:06:49.112119 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72675b-da4d-44ee-a38d-bdec77cee91c service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:06:49.121627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 775/1524] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:06:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:06:49.950722 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7c974-fa0f-4e4b-9495-6a0bb2a29c3e None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:49.951354 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7c974-fa0f-4e4b-9495-6a0bb2a29c3e None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:49.951354 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7c974-fa0f-4e4b-9495-6a0bb2a29c3e None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:49.952656 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 750/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:06:49.957472 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44151f62-607b-43ec-bc02-2b1f26ca6cb0 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:49.957895 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44151f62-607b-43ec-bc02-2b1f26ca6cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:49.958012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44151f62-607b-43ec-bc02-2b1f26ca6cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:49.977664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-44151f62-607b-43ec-bc02-2b1f26ca6cb0 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:06:49.978790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-44151f62-607b-43ec-bc02-2b1f26ca6cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:06:49.987362 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 776/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:06:50.141191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7c974-fa0f-4e4b-9495-6a0bb2a29c3e None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:06:50.155593 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7c974-fa0f-4e4b-9495-6a0bb2a29c3e None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:06:50.161639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7512287-18df-4057-be56-4788c9713a8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:06:50.162232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7512287-18df-4057-be56-4788c9713a8c service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:50.162510 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7512287-18df-4057-be56-4788c9713a8c service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:50.162782 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7512287-18df-4057-be56-4788c9713a8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:50.171121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7512287-18df-4057-be56-4788c9713a8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:06:50.173216 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7512287-18df-4057-be56-4788c9713a8c service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:06:50.182455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 751/1527] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:06:53.985175 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdab921-0d4d-41c0-a3b9-1e067c9c4f43 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:53.985792 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdab921-0d4d-41c0-a3b9-1e067c9c4f43 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:53.985792 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdab921-0d4d-41c0-a3b9-1e067c9c4f43 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:53.987101 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 777/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:06:53.992166 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9e3d0-8c56-4f20-a001-9cd6787b5d1e None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:53.992604 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9e3d0-8c56-4f20-a001-9cd6787b5d1e None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:53.992717 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb9e3d0-8c56-4f20-a001-9cd6787b5d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:54.011458 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-dfb9e3d0-8c56-4f20-a001-9cd6787b5d1e None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:06:54.012635 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-dfb9e3d0-8c56-4f20-a001-9cd6787b5d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:06:54.020750 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 752/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:06:54.302023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdab921-0d4d-41c0-a3b9-1e067c9c4f43 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:06:54.304115 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-7bdab921-0d4d-41c0-a3b9-1e067c9c4f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:06:54.385700 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdab921-0d4d-41c0-a3b9-1e067c9c4f43 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:06:54.389297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a8f90-f3d5-4a08-ac19-57b8b146b897 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:06:54.389899 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a8f90-f3d5-4a08-ac19-57b8b146b897 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:54.390176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a8f90-f3d5-4a08-ac19-57b8b146b897 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:54.390443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a8f90-f3d5-4a08-ac19-57b8b146b897 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:54.400118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a8f90-f3d5-4a08-ac19-57b8b146b897 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:06:54.401546 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363a8f90-f3d5-4a08-ac19-57b8b146b897 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:06:54.410858 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 778/1530] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:06:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:06:56.292596 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e688fd-f65d-450d-8a0a-96ecee8a1f16 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:56.293161 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e688fd-f65d-450d-8a0a-96ecee8a1f16 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:56.293161 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e688fd-f65d-450d-8a0a-96ecee8a1f16 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:56.294477 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 753/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:06:56.299764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777082c-90b1-4cc1-9cfc-500259ea4311 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:56.300161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777082c-90b1-4cc1-9cfc-500259ea4311 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:56.300328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777082c-90b1-4cc1-9cfc-500259ea4311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:56.319352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-9777082c-90b1-4cc1-9cfc-500259ea4311 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:06:56.320510 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-9777082c-90b1-4cc1-9cfc-500259ea4311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:06:56.328126 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 779/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:06:56.611208 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e688fd-f65d-450d-8a0a-96ecee8a1f16 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:06:56.612970 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-f6e688fd-f65d-450d-8a0a-96ecee8a1f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:06:56.626224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e688fd-f65d-450d-8a0a-96ecee8a1f16 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:06:56.629684 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c456280-04ea-4072-8652-c000058bbd7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:06:56.630272 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c456280-04ea-4072-8652-c000058bbd7c service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:06:56.630548 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c456280-04ea-4072-8652-c000058bbd7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:06:56.630821 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c456280-04ea-4072-8652-c000058bbd7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:06:56.639457 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c456280-04ea-4072-8652-c000058bbd7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:06:56.640865 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c456280-04ea-4072-8652-c000058bbd7c service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:06:56.650565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 754/1533] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:06:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:07:00.228170 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330a5f9-fc84-4210-b305-b22d9f072efb None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:07:00.228675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330a5f9-fc84-4210-b305-b22d9f072efb None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:07:00.228778 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330a5f9-fc84-4210-b305-b22d9f072efb None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:07:00.230054 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 780/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:07:00.235271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a954bb-ea6b-40d4-b97e-db233ab87a4e None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:07:00.235733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a954bb-ea6b-40d4-b97e-db233ab87a4e None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:07:00.235832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a954bb-ea6b-40d4-b97e-db233ab87a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:07:00.243231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=81 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:07:00.256450 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-33a954bb-ea6b-40d4-b97e-db233ab87a4e None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:07:00.257647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-33a954bb-ea6b-40d4-b97e-db233ab87a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:07:00.265553 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 755/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:07:00.816472 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330a5f9-fc84-4210-b305-b22d9f072efb None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:07:00.818127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6330a5f9-fc84-4210-b305-b22d9f072efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:07:00.882414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:07:00.896042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330a5f9-fc84-4210-b305-b22d9f072efb None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:07:00.899514 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f27a35c-8015-4e46-897e-c1e27d1d777b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:07:00.900103 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27a35c-8015-4e46-897e-c1e27d1d777b service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:07:00.900375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27a35c-8015-4e46-897e-c1e27d1d777b service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:07:00.900643 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f27a35c-8015-4e46-897e-c1e27d1d777b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:07:00.910108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f27a35c-8015-4e46-897e-c1e27d1d777b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:07:00.911579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f27a35c-8015-4e46-897e-c1e27d1d777b service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:07:00.920710 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 781/1536] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:07:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:07:02.833963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2e516-e43b-4fb1-8ac7-2a674d128645 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:07:02.834550 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2e516-e43b-4fb1-8ac7-2a674d128645 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:07:02.834550 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b2e516-e43b-4fb1-8ac7-2a674d128645 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:07:02.835886 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 756/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:07:02.840925 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade06792-648a-409f-89b3-e1bdf618684d None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:07:02.841366 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade06792-648a-409f-89b3-e1bdf618684d None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:07:02.841472 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade06792-648a-409f-89b3-e1bdf618684d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:07:02.861383 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-ade06792-648a-409f-89b3-e1bdf618684d None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:07:02.862568 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-ade06792-648a-409f-89b3-e1bdf618684d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:07:02.870196 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 782/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:07:03.427044 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2e516-e43b-4fb1-8ac7-2a674d128645 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:07:03.428718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-01b2e516-e43b-4fb1-8ac7-2a674d128645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:07:03.442024 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b2e516-e43b-4fb1-8ac7-2a674d128645 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:07:03.445870 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6599f334-a23e-41e5-91dd-03bce39a74de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:07:03.446591 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599f334-a23e-41e5-91dd-03bce39a74de service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:07:03.446750 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599f334-a23e-41e5-91dd-03bce39a74de service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:07:03.447017 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599f334-a23e-41e5-91dd-03bce39a74de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:07:03.456412 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6599f334-a23e-41e5-91dd-03bce39a74de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:07:03.457827 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6599f334-a23e-41e5-91dd-03bce39a74de service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:07:03.467171 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 757/1539] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:07:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:07:12.856405 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=11 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:07:13.462913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=8 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:25.872030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec9204-67ec-412a-80ad-0d98062821d2 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:25.872689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec9204-67ec-412a-80ad-0d98062821d2 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:25.872689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec9204-67ec-412a-80ad-0d98062821d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:25.895943 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-74ec9204-67ec-412a-80ad-0d98062821d2 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:25.897522 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-74ec9204-67ec-412a-80ad-0d98062821d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:25.909416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 783/1540] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:25.918781 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a18b2-3b93-4edd-af52-e98501cd38e4 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:25.919378 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a18b2-3b93-4edd-af52-e98501cd38e4 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:25.919378 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a18b2-3b93-4edd-af52-e98501cd38e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:25.951860 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-8e6a18b2-3b93-4edd-af52-e98501cd38e4 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:25.953334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-8e6a18b2-3b93-4edd-af52-e98501cd38e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:25.963725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 758/1541] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:25.968276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec9204-67ec-412a-80ad-0d98062821d2 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:25.978767 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec9204-67ec-412a-80ad-0d98062821d2 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:25.989827 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e5c1ef-92db-4127-8c86-227aef6b5a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:25.990788 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5c1ef-92db-4127-8c86-227aef6b5a1d admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:25.995532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5c1ef-92db-4127-8c86-227aef6b5a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:25.996161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e5c1ef-92db-4127-8c86-227aef6b5a1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:25.997102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5c1ef-92db-4127-8c86-227aef6b5a1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:08:25.997631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5c1ef-92db-4127-8c86-227aef6b5a1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:25.999615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e5c1ef-92db-4127-8c86-227aef6b5a1d admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:26.012982 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 784/1542] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:08:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:08:26.023928 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a18b2-3b93-4edd-af52-e98501cd38e4 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:26.034644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a18b2-3b93-4edd-af52-e98501cd38e4 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:26.038829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4d7ff-f7b5-42ef-9f2b-3998c70df1d6 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.039464 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4d7ff-f7b5-42ef-9f2b-3998c70df1d6 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.039986 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4d7ff-f7b5-42ef-9f2b-3998c70df1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.055093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5307eaf-43d2-48ba-a86d-d76411d270c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:26.055093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5307eaf-43d2-48ba-a86d-d76411d270c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.055093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5307eaf-43d2-48ba-a86d-d76411d270c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.055093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5307eaf-43d2-48ba-a86d-d76411d270c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.055736 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5307eaf-43d2-48ba-a86d-d76411d270c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:26.057641 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5307eaf-43d2-48ba-a86d-d76411d270c0 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:26.092733 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-93f4d7ff-f7b5-42ef-9f2b-3998c70df1d6 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:26.093956 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-93f4d7ff-f7b5-42ef-9f2b-3998c70df1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:26.117472 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 785/1543] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:08:26.126826 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aeccba-92cc-4e91-8d1d-8ef7975cb218 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.127026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aeccba-92cc-4e91-8d1d-8ef7975cb218 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.127801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aeccba-92cc-4e91-8d1d-8ef7975cb218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.139680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 759/1544] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:08:26 2022] POST /identity/v3/projects => generated 367 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:26.157216 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5307eaf-43d2-48ba-a86d-d76411d270c0 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:26.163518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-a3aeccba-92cc-4e91-8d1d-8ef7975cb218 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:26.165028 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a3aeccba-92cc-4e91-8d1d-8ef7975cb218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:26.284192 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5307eaf-43d2-48ba-a86d-d76411d270c0 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:26.293000 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 786/1545] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:08:26.294656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbbe54-d8f9-4cf5-9d9a-a765b5f4b1bf None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.295256 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbbe54-d8f9-4cf5-9d9a-a765b5f4b1bf None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.295738 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbbe54-d8f9-4cf5-9d9a-a765b5f4b1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.306941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1bed8-2386-47ae-9923-0f7bc97ceb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:26.306941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1bed8-2386-47ae-9923-0f7bc97ceb2e admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.306941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1bed8-2386-47ae-9923-0f7bc97ceb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.306941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1bed8-2386-47ae-9923-0f7bc97ceb2e admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.307438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1bed8-2386-47ae-9923-0f7bc97ceb2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:26.307438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: WARNING py.warnings [None req-e4d1bed8-2386-47ae-9923-0f7bc97ceb2e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:08:26.307438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: warnings.warn(msg) Sep 02 09:08:26.307438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: Sep 02 09:08:26.307438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1bed8-2386-47ae-9923-0f7bc97ceb2e admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:26.337619 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-39cbbe54-d8f9-4cf5-9d9a-a765b5f4b1bf None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:26.342955 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-39cbbe54-d8f9-4cf5-9d9a-a765b5f4b1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:26.352848 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 760/1546] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:08:26 2022] POST /identity/v3/users => generated 284 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:26.353936 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac604ee-15c3-4da9-ade0-ad7e3377ff3f None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.354223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac604ee-15c3-4da9-ade0-ad7e3377ff3f None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.354497 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac604ee-15c3-4da9-ade0-ad7e3377ff3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.373634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 787/1547] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:26.374826 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbbe54-d8f9-4cf5-9d9a-a765b5f4b1bf None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:26.395324 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbbe54-d8f9-4cf5-9d9a-a765b5f4b1bf None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:26.396157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-fac604ee-15c3-4da9-ade0-ad7e3377ff3f None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:26.398518 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-fac604ee-15c3-4da9-ade0-ad7e3377ff3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:26.406087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa5265-3ab2-4434-a2c3-5f2249cd31f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:26.408798 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa5265-3ab2-4434-a2c3-5f2249cd31f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.409269 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa5265-3ab2-4434-a2c3-5f2249cd31f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.409863 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa5265-3ab2-4434-a2c3-5f2249cd31f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.411985 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa5265-3ab2-4434-a2c3-5f2249cd31f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:08:26.412451 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa5265-3ab2-4434-a2c3-5f2249cd31f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:26.413938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 761/1548] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:26.416503 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac604ee-15c3-4da9-ade0-ad7e3377ff3f None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:26.416948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa5265-3ab2-4434-a2c3-5f2249cd31f1 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:26.435599 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac604ee-15c3-4da9-ade0-ad7e3377ff3f None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:26.440036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 788/1549] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:08:26.441902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad02438-c9e1-462d-834b-6bd99cc4a2e1 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.442682 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad02438-c9e1-462d-834b-6bd99cc4a2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.443654 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad02438-c9e1-462d-834b-6bd99cc4a2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.449502 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b9a17-1f31-4d79-a4ce-01bd2b949755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:26.450320 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b9a17-1f31-4d79-a4ce-01bd2b949755 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.450779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b9a17-1f31-4d79-a4ce-01bd2b949755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.451231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b9a17-1f31-4d79-a4ce-01bd2b949755 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.452047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b9a17-1f31-4d79-a4ce-01bd2b949755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:26.453827 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b9a17-1f31-4d79-a4ce-01bd2b949755 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:26.471048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 762/1550] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:26.472443 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688c59c-13cd-4eb0-addc-acf33b4e3527 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.472728 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688c59c-13cd-4eb0-addc-acf33b4e3527 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.472997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688c59c-13cd-4eb0-addc-acf33b4e3527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.481331 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-3ad02438-c9e1-462d-834b-6bd99cc4a2e1 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:26.483772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-3ad02438-c9e1-462d-834b-6bd99cc4a2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:26.493805 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 789/1551] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:26.494041 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-4688c59c-13cd-4eb0-addc-acf33b4e3527 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:26.498508 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-4688c59c-13cd-4eb0-addc-acf33b4e3527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:26.499511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead9f6b-3404-4cf4-a9d3-53215c26d33e None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.499979 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead9f6b-3404-4cf4-a9d3-53215c26d33e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.500429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead9f6b-3404-4cf4-a9d3-53215c26d33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.507333 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 763/1552] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:26.508383 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688c59c-13cd-4eb0-addc-acf33b4e3527 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:26.519702 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688c59c-13cd-4eb0-addc-acf33b4e3527 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:26.523767 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-2ead9f6b-3404-4cf4-a9d3-53215c26d33e None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:26.525251 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-2ead9f6b-3404-4cf4-a9d3-53215c26d33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:26.525545 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e284e-78a7-4171-899c-6fa2cb7f0d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:26.526231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e284e-78a7-4171-899c-6fa2cb7f0d23 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.526604 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e284e-78a7-4171-899c-6fa2cb7f0d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.526914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e284e-78a7-4171-899c-6fa2cb7f0d23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.527643 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e284e-78a7-4171-899c-6fa2cb7f0d23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:26.530064 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95e284e-78a7-4171-899c-6fa2cb7f0d23 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:26.535330 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 790/1553] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:26.537242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ae061-77ec-4054-9016-4ce3bb5e97f1 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.538273 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ae061-77ec-4054-9016-4ce3bb5e97f1 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.538778 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ae061-77ec-4054-9016-4ce3bb5e97f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.563303 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-6c7ae061-77ec-4054-9016-4ce3bb5e97f1 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:26.564657 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6c7ae061-77ec-4054-9016-4ce3bb5e97f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:26.570870 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 764/1554] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:08:26 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:26.571668 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e284e-78a7-4171-899c-6fa2cb7f0d23 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:26.573286 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 791/1555] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:26.574215 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ae061-77ec-4054-9016-4ce3bb5e97f1 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:26.584290 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ae061-77ec-4054-9016-4ce3bb5e97f1 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:26.634018 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e284e-78a7-4171-899c-6fa2cb7f0d23 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:26.638713 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a13694-6589-41e5-aad9-43f2ccb7f261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:26.639418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a13694-6589-41e5-aad9-43f2ccb7f261 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.639698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a13694-6589-41e5-aad9-43f2ccb7f261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.639971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a13694-6589-41e5-aad9-43f2ccb7f261 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.640080 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63744b99-7032-4dbb-9057-b6eeb9845b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:26.640612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a13694-6589-41e5-aad9-43f2ccb7f261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:26.640735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63744b99-7032-4dbb-9057-b6eeb9845b7d admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.640926 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63744b99-7032-4dbb-9057-b6eeb9845b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.641200 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63744b99-7032-4dbb-9057-b6eeb9845b7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.642352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63744b99-7032-4dbb-9057-b6eeb9845b7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:08:26.642659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63744b99-7032-4dbb-9057-b6eeb9845b7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:26.642776 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a13694-6589-41e5-aad9-43f2ccb7f261 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:26.644282 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63744b99-7032-4dbb-9057-b6eeb9845b7d admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:26.652541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 765/1556] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:26.653715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7e3e4-3a00-42c6-9ed4-d679cb32691e None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.654002 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7e3e4-3a00-42c6-9ed4-d679cb32691e None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.654273 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7e3e4-3a00-42c6-9ed4-d679cb32691e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.662651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 792/1557] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:08:26.662651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63744b99-7032-4dbb-9057-b6eeb9845b7d admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:26.662903 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:26.672986 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63744b99-7032-4dbb-9057-b6eeb9845b7d admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:26.675774 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-08b7e3e4-3a00-42c6-9ed4-d679cb32691e None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:26.676982 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-08b7e3e4-3a00-42c6-9ed4-d679cb32691e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:26.678085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3854a432-8127-4076-a61a-c66c7828af43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:26.678723 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854a432-8127-4076-a61a-c66c7828af43 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.678946 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854a432-8127-4076-a61a-c66c7828af43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.679261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3854a432-8127-4076-a61a-c66c7828af43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.679878 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854a432-8127-4076-a61a-c66c7828af43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:08:26.680176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854a432-8127-4076-a61a-c66c7828af43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:26.681765 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3854a432-8127-4076-a61a-c66c7828af43 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:26.684963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 766/1558] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:26.686219 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e6d43-8204-4fd6-8d9a-fe85b728c8c2 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.686507 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e6d43-8204-4fd6-8d9a-fe85b728c8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.686782 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e6d43-8204-4fd6-8d9a-fe85b728c8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.701163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 793/1559] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:08:26.701163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e919e-cc19-4022-8d71-1c149f8d9db0 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.701163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e919e-cc19-4022-8d71-1c149f8d9db0 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.701163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e919e-cc19-4022-8d71-1c149f8d9db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.708979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-ca1e6d43-8204-4fd6-8d9a-fe85b728c8c2 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:26.710163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ca1e6d43-8204-4fd6-8d9a-fe85b728c8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:26.718053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-701e919e-cc19-4022-8d71-1c149f8d9db0 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:26.718476 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 767/1560] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:26.719244 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-701e919e-cc19-4022-8d71-1c149f8d9db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:26.719376 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e6d43-8204-4fd6-8d9a-fe85b728c8c2 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:26.728695 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 794/1561] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:26.729494 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e6d43-8204-4fd6-8d9a-fe85b728c8c2 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:26.730027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1f2ef-709a-40be-a0c5-0ad0bcba43c3 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.730292 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1f2ef-709a-40be-a0c5-0ad0bcba43c3 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.730571 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1f2ef-709a-40be-a0c5-0ad0bcba43c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.734285 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da629fec-cb69-49d5-9be8-d6999be8aec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:26.735099 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da629fec-cb69-49d5-9be8-d6999be8aec1 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.735381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da629fec-cb69-49d5-9be8-d6999be8aec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.735931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da629fec-cb69-49d5-9be8-d6999be8aec1 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.736411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da629fec-cb69-49d5-9be8-d6999be8aec1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:26.738298 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da629fec-cb69-49d5-9be8-d6999be8aec1 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:26.750444 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-7fe1f2ef-709a-40be-a0c5-0ad0bcba43c3 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:26.752414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-7fe1f2ef-709a-40be-a0c5-0ad0bcba43c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:26.760006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 768/1562] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:08:26 2022] POST /identity/v3/users => generated 289 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:26.761064 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 795/1563] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:26.761863 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1f2ef-709a-40be-a0c5-0ad0bcba43c3 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:26.763737 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da629fec-cb69-49d5-9be8-d6999be8aec1 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:26.766801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:26.773338 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1f2ef-709a-40be-a0c5-0ad0bcba43c3 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:26.773957 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da629fec-cb69-49d5-9be8-d6999be8aec1 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:26.778779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf7888-c196-42d2-ac49-6cce13d81a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:26.779107 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ad2bc-b8ee-45ed-931f-cddae63645d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:26.779533 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf7888-c196-42d2-ac49-6cce13d81a25 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.779746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ad2bc-b8ee-45ed-931f-cddae63645d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.779937 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf7888-c196-42d2-ac49-6cce13d81a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.780050 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ad2bc-b8ee-45ed-931f-cddae63645d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.780212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf7888-c196-42d2-ac49-6cce13d81a25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.780325 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ad2bc-b8ee-45ed-931f-cddae63645d8 admin admin] PATH_INFO: `/v3/projects/a256f597167b463a95ceadb3eca9c0a3/users/5778878319b9470eb17a52d4af060b37/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.780839 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf7888-c196-42d2-ac49-6cce13d81a25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:26.781865 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: WARNING py.warnings [None req-31bf7888-c196-42d2-ac49-6cce13d81a25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:08:26.781865 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: warnings.warn(msg) Sep 02 09:08:26.781865 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: Sep 02 09:08:26.783883 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf7888-c196-42d2-ac49-6cce13d81a25 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:26.794435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ad2bc-b8ee-45ed-931f-cddae63645d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a256f597167b463a95ceadb3eca9c0a3, user_id=5778878319b9470eb17a52d4af060b37, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:26.797110 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ad2bc-b8ee-45ed-931f-cddae63645d8 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:26.810880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 769/1564] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:08:26 2022] PUT /identity/v3/projects/a256f597167b463a95ceadb3eca9c0a3/users/5778878319b9470eb17a52d4af060b37/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:26.811031 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 796/1565] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:08:26 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:26.815468 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf7888-c196-42d2-ac49-6cce13d81a25 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:26.816365 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ad2bc-b8ee-45ed-931f-cddae63645d8 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:26.879413 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ad2bc-b8ee-45ed-931f-cddae63645d8 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:26.879655 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf7888-c196-42d2-ac49-6cce13d81a25 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:26.884176 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e7589-e651-4cf3-b066-02ddbbf57b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:26.884354 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea0d28-c191-4c28-a6c1-47cf15c8f8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:26.884787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e7589-e651-4cf3-b066-02ddbbf57b77 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.884939 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea0d28-c191-4c28-a6c1-47cf15c8f8d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.885070 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e7589-e651-4cf3-b066-02ddbbf57b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.885171 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea0d28-c191-4c28-a6c1-47cf15c8f8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.885334 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea0d28-c191-4c28-a6c1-47cf15c8f8d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.885430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e7589-e651-4cf3-b066-02ddbbf57b77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.885951 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea0d28-c191-4c28-a6c1-47cf15c8f8d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:26.886064 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e7589-e651-4cf3-b066-02ddbbf57b77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:08:26.886273 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e7589-e651-4cf3-b066-02ddbbf57b77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:26.887314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: WARNING py.warnings [None req-741e7589-e651-4cf3-b066-02ddbbf57b77 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:08:26.887314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: warnings.warn(msg) Sep 02 09:08:26.887314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: Sep 02 09:08:26.887918 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea0d28-c191-4c28-a6c1-47cf15c8f8d4 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:26.888184 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e7589-e651-4cf3-b066-02ddbbf57b77 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:26.908525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 770/1566] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:08:26.908525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10694f3f-3c8c-44fd-8cb1-c635384571aa None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.908525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10694f3f-3c8c-44fd-8cb1-c635384571aa None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.908525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10694f3f-3c8c-44fd-8cb1-c635384571aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.915046 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 797/1567] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:08:26 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:26.916180 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b8056-0c3f-4079-ae0d-1f6fc6edc8d1 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.916465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b8056-0c3f-4079-ae0d-1f6fc6edc8d1 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.917017 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b8056-0c3f-4079-ae0d-1f6fc6edc8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:26.923779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-10694f3f-3c8c-44fd-8cb1-c635384571aa None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:26.924990 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-10694f3f-3c8c-44fd-8cb1-c635384571aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:26.935757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-443b8056-0c3f-4079-ae0d-1f6fc6edc8d1 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:26.936898 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-443b8056-0c3f-4079-ae0d-1f6fc6edc8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:26.982905 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 771/1568] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:26.983929 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10694f3f-3c8c-44fd-8cb1-c635384571aa None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:26.992870 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10694f3f-3c8c-44fd-8cb1-c635384571aa None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:26.995698 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 798/1569] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:26.996704 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b8056-0c3f-4079-ae0d-1f6fc6edc8d1 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:26.998688 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6eaf8-0e63-4aa8-92d9-623fd518a701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:26.999313 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6eaf8-0e63-4aa8-92d9-623fd518a701 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:26.999590 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6eaf8-0e63-4aa8-92d9-623fd518a701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:26.999851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6eaf8-0e63-4aa8-92d9-623fd518a701 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.000585 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6eaf8-0e63-4aa8-92d9-623fd518a701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.002705 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6eaf8-0e63-4aa8-92d9-623fd518a701 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.006215 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b8056-0c3f-4079-ae0d-1f6fc6edc8d1 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.010730 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9b497-fdba-4ce1-b768-0d1b898959bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.011331 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9b497-fdba-4ce1-b768-0d1b898959bd admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.011611 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9b497-fdba-4ce1-b768-0d1b898959bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.011881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9b497-fdba-4ce1-b768-0d1b898959bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.012581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 772/1570] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:27.012710 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9b497-fdba-4ce1-b768-0d1b898959bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:08:27.012871 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9b497-fdba-4ce1-b768-0d1b898959bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.013176 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6eaf8-0e63-4aa8-92d9-623fd518a701 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.014525 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9b497-fdba-4ce1-b768-0d1b898959bd admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.023031 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6eaf8-0e63-4aa8-92d9-623fd518a701 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.026877 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 799/1571] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:08:27.027593 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76e383-a6ba-456e-a140-7a8c8cda0505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.028094 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a247e4-7635-47b0-90b1-96ca28070a09 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.028186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76e383-a6ba-456e-a140-7a8c8cda0505 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.028380 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a247e4-7635-47b0-90b1-96ca28070a09 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.028469 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76e383-a6ba-456e-a140-7a8c8cda0505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.028656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a247e4-7635-47b0-90b1-96ca28070a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.028744 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76e383-a6ba-456e-a140-7a8c8cda0505 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.029290 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76e383-a6ba-456e-a140-7a8c8cda0505 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.031202 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76e383-a6ba-456e-a140-7a8c8cda0505 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.052063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 773/1572] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:08:27 2022] POST /identity/v3/users => generated 289 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:27.052851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76e383-a6ba-456e-a140-7a8c8cda0505 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.058467 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-85a247e4-7635-47b0-90b1-96ca28070a09 None None] MFA Rules not processed for user `5778878319b9470eb17a52d4af060b37`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:27.059950 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-85a247e4-7635-47b0-90b1-96ca28070a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:27.065228 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76e383-a6ba-456e-a140-7a8c8cda0505 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.069819 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e257d98-1c0d-46a7-b049-72209810e632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.070675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e257d98-1c0d-46a7-b049-72209810e632 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.070961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e257d98-1c0d-46a7-b049-72209810e632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.071266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e257d98-1c0d-46a7-b049-72209810e632 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.071889 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e257d98-1c0d-46a7-b049-72209810e632 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.073787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e257d98-1c0d-46a7-b049-72209810e632 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.085695 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:27.102424 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 774/1573] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:08:27 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:27.103299 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e257d98-1c0d-46a7-b049-72209810e632 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.104890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:27.130550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 800/1574] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:27.131380 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a247e4-7635-47b0-90b1-96ca28070a09 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.144550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a247e4-7635-47b0-90b1-96ca28070a09 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.166107 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e257d98-1c0d-46a7-b049-72209810e632 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.170983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c502cf-fd8f-459f-ad6e-0144715f88ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.171619 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c502cf-fd8f-459f-ad6e-0144715f88ef admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.171931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c502cf-fd8f-459f-ad6e-0144715f88ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.172250 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c502cf-fd8f-459f-ad6e-0144715f88ef admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.172962 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c502cf-fd8f-459f-ad6e-0144715f88ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.175558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c502cf-fd8f-459f-ad6e-0144715f88ef admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.197349 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 775/1575] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:08:27 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:27.200026 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c502cf-fd8f-459f-ad6e-0144715f88ef admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.209746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c502cf-fd8f-459f-ad6e-0144715f88ef admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.212736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e898c99-2ee2-428d-9932-11aa0f542368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.213345 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e898c99-2ee2-428d-9932-11aa0f542368 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.213625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e898c99-2ee2-428d-9932-11aa0f542368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.213892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e898c99-2ee2-428d-9932-11aa0f542368 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.214524 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e898c99-2ee2-428d-9932-11aa0f542368 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:08:27.214820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e898c99-2ee2-428d-9932-11aa0f542368 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.215051 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b69a8-286b-47ea-b017-a1102a5506c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.215764 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b69a8-286b-47ea-b017-a1102a5506c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.216144 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b69a8-286b-47ea-b017-a1102a5506c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.216518 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b69a8-286b-47ea-b017-a1102a5506c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.216647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e898c99-2ee2-428d-9932-11aa0f542368 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.217334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b69a8-286b-47ea-b017-a1102a5506c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:08:27.217737 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b69a8-286b-47ea-b017-a1102a5506c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.219411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b69a8-286b-47ea-b017-a1102a5506c1 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.228137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 801/1576] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:08:27.237096 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 776/1577] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:08:27.237096 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b69a8-286b-47ea-b017-a1102a5506c1 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.237384 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e898c99-2ee2-428d-9932-11aa0f542368 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.245300 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b69a8-286b-47ea-b017-a1102a5506c1 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.246413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e898c99-2ee2-428d-9932-11aa0f542368 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.249929 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a52c41-28fd-4e29-8c4b-7d7570348993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.250509 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a52c41-28fd-4e29-8c4b-7d7570348993 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.250789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a52c41-28fd-4e29-8c4b-7d7570348993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.250898 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa691c03-c93b-4a7d-a9dd-e49e62db6675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.251063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a52c41-28fd-4e29-8c4b-7d7570348993 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.252054 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa691c03-c93b-4a7d-a9dd-e49e62db6675 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.252332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa691c03-c93b-4a7d-a9dd-e49e62db6675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.252445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a52c41-28fd-4e29-8c4b-7d7570348993 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.252607 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa691c03-c93b-4a7d-a9dd-e49e62db6675 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.253302 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa691c03-c93b-4a7d-a9dd-e49e62db6675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.254240 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a52c41-28fd-4e29-8c4b-7d7570348993 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.254857 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa691c03-c93b-4a7d-a9dd-e49e62db6675 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.265579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 802/1578] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:27.266442 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa691c03-c93b-4a7d-a9dd-e49e62db6675 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.276925 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa691c03-c93b-4a7d-a9dd-e49e62db6675 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.280808 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 777/1579] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:08:27 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:27.281466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a52c41-28fd-4e29-8c4b-7d7570348993 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.282534 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcab66-8865-4e2f-a661-d843d5c86f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.283234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcab66-8865-4e2f-a661-d843d5c86f02 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.283642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcab66-8865-4e2f-a661-d843d5c86f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.284097 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcab66-8865-4e2f-a661-d843d5c86f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.284741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efcab66-8865-4e2f-a661-d843d5c86f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.286375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efcab66-8865-4e2f-a661-d843d5c86f02 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.299916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 803/1580] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:27.299916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcab66-8865-4e2f-a661-d843d5c86f02 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.344040 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a52c41-28fd-4e29-8c4b-7d7570348993 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.349752 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab8c49-f0b6-4b03-afb7-003be8e4e026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.350377 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab8c49-f0b6-4b03-afb7-003be8e4e026 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.350660 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab8c49-f0b6-4b03-afb7-003be8e4e026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.350937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab8c49-f0b6-4b03-afb7-003be8e4e026 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.351615 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab8c49-f0b6-4b03-afb7-003be8e4e026 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.353573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab8c49-f0b6-4b03-afb7-003be8e4e026 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.358832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcab66-8865-4e2f-a661-d843d5c86f02 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.375969 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 778/1581] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:08:27 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:27.377522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab8c49-f0b6-4b03-afb7-003be8e4e026 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.386943 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab8c49-f0b6-4b03-afb7-003be8e4e026 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.391863 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53eb6c-b3bb-453f-8ca8-c5fde1d0cb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.392462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53eb6c-b3bb-453f-8ca8-c5fde1d0cb67 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.392749 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53eb6c-b3bb-453f-8ca8-c5fde1d0cb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.393154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53eb6c-b3bb-453f-8ca8-c5fde1d0cb67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.393798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53eb6c-b3bb-453f-8ca8-c5fde1d0cb67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.395428 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53eb6c-b3bb-453f-8ca8-c5fde1d0cb67 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.406221 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 779/1582] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:27.407012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53eb6c-b3bb-453f-8ca8-c5fde1d0cb67 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.413652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5da76f-9892-49a3-ac97-086e48857aac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.414255 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5da76f-9892-49a3-ac97-086e48857aac service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.414552 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5da76f-9892-49a3-ac97-086e48857aac service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.414818 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5da76f-9892-49a3-ac97-086e48857aac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.418347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53eb6c-b3bb-453f-8ca8-c5fde1d0cb67 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.422874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07296163-f67b-44c8-802d-fc26208f38c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.423998 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07296163-f67b-44c8-802d-fc26208f38c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.424292 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07296163-f67b-44c8-802d-fc26208f38c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.424566 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07296163-f67b-44c8-802d-fc26208f38c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.425182 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07296163-f67b-44c8-802d-fc26208f38c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.427049 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07296163-f67b-44c8-802d-fc26208f38c7 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.440175 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:27.454207 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 780/1583] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:08:27 2022] POST /identity/v3/projects => generated 393 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:27.454989 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07296163-f67b-44c8-802d-fc26208f38c7 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.459345 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:27.474927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5da76f-9892-49a3-ac97-086e48857aac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.476475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5da76f-9892-49a3-ac97-086e48857aac service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.485222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 804/1584] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:08:27.486581 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5da76f-9892-49a3-ac97-086e48857aac service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.496604 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5da76f-9892-49a3-ac97-086e48857aac service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.522889 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07296163-f67b-44c8-802d-fc26208f38c7 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.527600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ca89b-7f65-4689-b767-85f8c582e8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.528210 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ca89b-7f65-4689-b767-85f8c582e8f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.528489 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ca89b-7f65-4689-b767-85f8c582e8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.528758 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806ca89b-7f65-4689-b767-85f8c582e8f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.529381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ca89b-7f65-4689-b767-85f8c582e8f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.531284 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806ca89b-7f65-4689-b767-85f8c582e8f6 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.560257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 781/1585] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:08:27 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:27.561074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ca89b-7f65-4689-b767-85f8c582e8f6 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.567835 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44cb97-bca2-45e0-b6b4-234fef6dc0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.568762 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44cb97-bca2-45e0-b6b4-234fef6dc0d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.569046 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44cb97-bca2-45e0-b6b4-234fef6dc0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.569485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa44cb97-bca2-45e0-b6b4-234fef6dc0d3 admin admin] PATH_INFO: `/v3/projects/8ebe6b70a4374511b388d20265e4ccdf/users/3adaaf8d979d4cf28456812f9dcfa809/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.588569 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44cb97-bca2-45e0-b6b4-234fef6dc0d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ebe6b70a4374511b388d20265e4ccdf, user_id=3adaaf8d979d4cf28456812f9dcfa809, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.589630 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: WARNING py.warnings [None req-aa44cb97-bca2-45e0-b6b4-234fef6dc0d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:08:27.589630 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: warnings.warn(msg) Sep 02 09:08:27.589630 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: Sep 02 09:08:27.592201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa44cb97-bca2-45e0-b6b4-234fef6dc0d3 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.614060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 805/1586] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:08:27 2022] PUT /identity/v3/projects/8ebe6b70a4374511b388d20265e4ccdf/users/3adaaf8d979d4cf28456812f9dcfa809/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:27.614060 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44cb97-bca2-45e0-b6b4-234fef6dc0d3 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.638717 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806ca89b-7f65-4689-b767-85f8c582e8f6 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.676608 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa44cb97-bca2-45e0-b6b4-234fef6dc0d3 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.681302 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935de20-668c-4398-9003-a33f40e0254d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.681963 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935de20-668c-4398-9003-a33f40e0254d admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.682248 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935de20-668c-4398-9003-a33f40e0254d admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.682635 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935de20-668c-4398-9003-a33f40e0254d admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.683361 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935de20-668c-4398-9003-a33f40e0254d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.685014 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935de20-668c-4398-9003-a33f40e0254d admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.698000 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 806/1587] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:27.698830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935de20-668c-4398-9003-a33f40e0254d admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.699733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2bbcb3-16ea-4a54-b9b1-a4e2a3d6a11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.700409 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2bbcb3-16ea-4a54-b9b1-a4e2a3d6a11a admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.700752 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2bbcb3-16ea-4a54-b9b1-a4e2a3d6a11a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.701081 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2bbcb3-16ea-4a54-b9b1-a4e2a3d6a11a admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.702044 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2bbcb3-16ea-4a54-b9b1-a4e2a3d6a11a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.704041 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2bbcb3-16ea-4a54-b9b1-a4e2a3d6a11a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.708887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935de20-668c-4398-9003-a33f40e0254d admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.714746 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21684a9-865f-4390-847f-8b0a245e61d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.715811 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21684a9-865f-4390-847f-8b0a245e61d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.716366 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21684a9-865f-4390-847f-8b0a245e61d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.716691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21684a9-865f-4390-847f-8b0a245e61d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.717392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21684a9-865f-4390-847f-8b0a245e61d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.719251 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21684a9-865f-4390-847f-8b0a245e61d3 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.727970 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 782/1588] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:08:27 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:27.728851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2bbcb3-16ea-4a54-b9b1-a4e2a3d6a11a admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.734661 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 807/1589] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:27.734661 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21684a9-865f-4390-847f-8b0a245e61d3 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.739028 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2bbcb3-16ea-4a54-b9b1-a4e2a3d6a11a admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.743845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d273a97-6162-4d62-859c-42d37753e23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.744603 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d273a97-6162-4d62-859c-42d37753e23e admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.745033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d273a97-6162-4d62-859c-42d37753e23e admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.745420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d273a97-6162-4d62-859c-42d37753e23e admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.745542 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21684a9-865f-4390-847f-8b0a245e61d3 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.746194 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d273a97-6162-4d62-859c-42d37753e23e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.747898 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d273a97-6162-4d62-859c-42d37753e23e admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.751573 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b644f-eeab-4e84-9449-689ed76afc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.752319 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b644f-eeab-4e84-9449-689ed76afc21 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.752666 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b644f-eeab-4e84-9449-689ed76afc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.753044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b644f-eeab-4e84-9449-689ed76afc21 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.754217 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b644f-eeab-4e84-9449-689ed76afc21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.756328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b644f-eeab-4e84-9449-689ed76afc21 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.758924 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 783/1590] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:27.759997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d273a97-6162-4d62-859c-42d37753e23e admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.770218 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d273a97-6162-4d62-859c-42d37753e23e admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.777141 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5dc695-60ec-41ab-a8d6-423a44781566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.777820 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5dc695-60ec-41ab-a8d6-423a44781566 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.778200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5dc695-60ec-41ab-a8d6-423a44781566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.778491 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5dc695-60ec-41ab-a8d6-423a44781566 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.779170 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5dc695-60ec-41ab-a8d6-423a44781566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.781578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5dc695-60ec-41ab-a8d6-423a44781566 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.786952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 808/1591] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:08:27 2022] POST /identity/v3/users => generated 297 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:27.788174 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfcfbff-e971-45a1-95bf-4ba270233840 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.788461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfcfbff-e971-45a1-95bf-4ba270233840 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.788736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfcfbff-e971-45a1-95bf-4ba270233840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.794176 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:27.808504 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 784/1592] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:08:27 2022] POST /identity/v3/users => generated 293 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:27.810132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-ccfcfbff-e971-45a1-95bf-4ba270233840 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:27.811633 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-ccfcfbff-e971-45a1-95bf-4ba270233840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:27.812581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5dc695-60ec-41ab-a8d6-423a44781566 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.819940 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 809/1593] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:27.820982 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfcfbff-e971-45a1-95bf-4ba270233840 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.827105 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5dc695-60ec-41ab-a8d6-423a44781566 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.830518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfcfbff-e971-45a1-95bf-4ba270233840 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.832063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92563f8-c53c-4a9e-a045-e5c0f0c03fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.832726 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92563f8-c53c-4a9e-a045-e5c0f0c03fa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.833046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92563f8-c53c-4a9e-a045-e5c0f0c03fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.833324 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92563f8-c53c-4a9e-a045-e5c0f0c03fa5 admin admin] PATH_INFO: `/v3/projects/2d6a6fe5db0a452a83dde4aac26b0161/users/4210f948dc7e4a5bac61d10eb6a62eee/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.836400 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921cb1a-6256-44e6-9f83-36c748be4781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.837407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921cb1a-6256-44e6-9f83-36c748be4781 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.837407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921cb1a-6256-44e6-9f83-36c748be4781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.837666 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921cb1a-6256-44e6-9f83-36c748be4781 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.838197 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c921cb1a-6256-44e6-9f83-36c748be4781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.839953 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c921cb1a-6256-44e6-9f83-36c748be4781 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.843096 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:27.850375 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92563f8-c53c-4a9e-a045-e5c0f0c03fa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d6a6fe5db0a452a83dde4aac26b0161, user_id=4210f948dc7e4a5bac61d10eb6a62eee, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.853527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92563f8-c53c-4a9e-a045-e5c0f0c03fa5 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.857264 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 810/1594] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:27.858475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921cb1a-6256-44e6-9f83-36c748be4781 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.868535 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 785/1595] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:08:27 2022] PUT /identity/v3/projects/2d6a6fe5db0a452a83dde4aac26b0161/users/4210f948dc7e4a5bac61d10eb6a62eee/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:27.869320 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92563f8-c53c-4a9e-a045-e5c0f0c03fa5 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.872764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921cb1a-6256-44e6-9f83-36c748be4781 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.935765 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58c1d5-3e41-431a-8242-f3e79051a45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.936417 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58c1d5-3e41-431a-8242-f3e79051a45a admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.936638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58c1d5-3e41-431a-8242-f3e79051a45a admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.936843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58c1d5-3e41-431a-8242-f3e79051a45a admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.937525 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d58c1d5-3e41-431a-8242-f3e79051a45a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.939273 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d58c1d5-3e41-431a-8242-f3e79051a45a admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.939961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92563f8-c53c-4a9e-a045-e5c0f0c03fa5 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.945228 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d9ce3-49be-4baf-9432-f692e4630a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.946141 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d9ce3-49be-4baf-9432-f692e4630a82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.946421 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d9ce3-49be-4baf-9432-f692e4630a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.946715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d9ce3-49be-4baf-9432-f692e4630a82 admin admin] PATH_INFO: `/v3/projects/6499e3f183d4412783693d05bd96c76d/users/24be3210cc564b869eaa76567ff83c63/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.953332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 811/1596] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:27.953332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58c1d5-3e41-431a-8242-f3e79051a45a admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:27.962466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085d9ce3-49be-4baf-9432-f692e4630a82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6499e3f183d4412783693d05bd96c76d, user_id=24be3210cc564b869eaa76567ff83c63, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.965328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085d9ce3-49be-4baf-9432-f692e4630a82 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.966507 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58c1d5-3e41-431a-8242-f3e79051a45a admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:27.971746 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19d496-efc4-4cbf-8c59-71aa9dc3e6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:27.972419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19d496-efc4-4cbf-8c59-71aa9dc3e6e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:27.972831 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19d496-efc4-4cbf-8c59-71aa9dc3e6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:27.973673 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd19d496-efc4-4cbf-8c59-71aa9dc3e6e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:27.974432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd19d496-efc4-4cbf-8c59-71aa9dc3e6e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:27.976073 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd19d496-efc4-4cbf-8c59-71aa9dc3e6e9 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:27.984232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 786/1597] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:08:27 2022] PUT /identity/v3/projects/6499e3f183d4412783693d05bd96c76d/users/24be3210cc564b869eaa76567ff83c63/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:27.985414 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d9ce3-49be-4baf-9432-f692e4630a82 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:28.000156 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 812/1598] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:28.002201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19d496-efc4-4cbf-8c59-71aa9dc3e6e9 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:28.067910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd19d496-efc4-4cbf-8c59-71aa9dc3e6e9 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:28.068784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d9ce3-49be-4baf-9432-f692e4630a82 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:28.078721 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae8904-8bfb-4e22-9732-ef589df2b26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:28.080196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae8904-8bfb-4e22-9732-ef589df2b26d admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:28.080621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae8904-8bfb-4e22-9732-ef589df2b26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:28.080921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae8904-8bfb-4e22-9732-ef589df2b26d admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:28.081878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae8904-8bfb-4e22-9732-ef589df2b26d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:28.084083 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae8904-8bfb-4e22-9732-ef589df2b26d admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:28.097102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 787/1599] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:28.098522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae8904-8bfb-4e22-9732-ef589df2b26d admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:28.113332 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae8904-8bfb-4e22-9732-ef589df2b26d admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:28.120694 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e86c6e-04b0-4e5f-ac4b-f1bdf51d63f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:28.121382 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e86c6e-04b0-4e5f-ac4b-f1bdf51d63f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:28.121854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e86c6e-04b0-4e5f-ac4b-f1bdf51d63f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:28.122189 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e86c6e-04b0-4e5f-ac4b-f1bdf51d63f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:28.122944 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e86c6e-04b0-4e5f-ac4b-f1bdf51d63f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:28.123804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9764b-e788-4c66-9f17-d95f5642e026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:28.124406 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9764b-e788-4c66-9f17-d95f5642e026 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:28.124689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9764b-e788-4c66-9f17-d95f5642e026 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:28.124963 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9764b-e788-4c66-9f17-d95f5642e026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:28.125068 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e86c6e-04b0-4e5f-ac4b-f1bdf51d63f2 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:28.134103 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9764b-e788-4c66-9f17-d95f5642e026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:28.135564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9764b-e788-4c66-9f17-d95f5642e026 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:28.136001 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 788/1600] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:28.140445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9dfd9-6fcf-4fc6-bbad-03696137fdcd None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:28.140846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9dfd9-6fcf-4fc6-bbad-03696137fdcd None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:28.141192 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9dfd9-6fcf-4fc6-bbad-03696137fdcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:28.145953 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 813/1601] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:08:28.146600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9764b-e788-4c66-9f17-d95f5642e026 service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:28.155924 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9764b-e788-4c66-9f17-d95f5642e026 service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:28.161509 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bff98a-6e6c-4a12-81e6-33a14f83e033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:28.162324 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bff98a-6e6c-4a12-81e6-33a14f83e033 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:28.162461 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-95f9dfd9-6fcf-4fc6-bbad-03696137fdcd None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:28.162604 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bff98a-6e6c-4a12-81e6-33a14f83e033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:28.162698 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bff98a-6e6c-4a12-81e6-33a14f83e033 admin admin] PATH_INFO: `/v3/projects/ced6a1e5137e41cf9f3c5935b2c65ae8/users/de7c42ad8d41425ba19af08fda44949f/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:28.163926 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-95f9dfd9-6fcf-4fc6-bbad-03696137fdcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:28.173935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 789/1602] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:28.177826 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9dfd9-6fcf-4fc6-bbad-03696137fdcd None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:28.179648 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bff98a-6e6c-4a12-81e6-33a14f83e033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ced6a1e5137e41cf9f3c5935b2c65ae8, user_id=de7c42ad8d41425ba19af08fda44949f, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:28.184625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bff98a-6e6c-4a12-81e6-33a14f83e033 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:28.188960 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9dfd9-6fcf-4fc6-bbad-03696137fdcd None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:28.195104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5b19e-189c-4656-a344-3f5140622ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:28.196261 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5b19e-189c-4656-a344-3f5140622ca9 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:28.196549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5b19e-189c-4656-a344-3f5140622ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:28.196872 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5b19e-189c-4656-a344-3f5140622ca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:28.197524 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c5b19e-189c-4656-a344-3f5140622ca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:28.199519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c5b19e-189c-4656-a344-3f5140622ca9 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:28.202057 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 814/1603] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:08:28 2022] PUT /identity/v3/projects/ced6a1e5137e41cf9f3c5935b2c65ae8/users/de7c42ad8d41425ba19af08fda44949f/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:28.203521 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369e2b4-6c22-4786-af10-c07106174246 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:28.203890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369e2b4-6c22-4786-af10-c07106174246 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:28.204256 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369e2b4-6c22-4786-af10-c07106174246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:28.225927 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 790/1604] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:28.225927 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5b19e-189c-4656-a344-3f5140622ca9 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:28.236661 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-9369e2b4-6c22-4786-af10-c07106174246 None None] MFA Rules not processed for user `24be3210cc564b869eaa76567ff83c63`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:28.237189 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5b19e-189c-4656-a344-3f5140622ca9 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:28.237858 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-9369e2b4-6c22-4786-af10-c07106174246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:28.244410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:28.263787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:28.313920 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a1088-fc5d-4f0c-96a0-f9b64943b40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:28.314681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a1088-fc5d-4f0c-96a0-f9b64943b40e admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:28.315035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a1088-fc5d-4f0c-96a0-f9b64943b40e admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:28.315405 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a1088-fc5d-4f0c-96a0-f9b64943b40e admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:28.316125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a1088-fc5d-4f0c-96a0-f9b64943b40e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:28.317818 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a1088-fc5d-4f0c-96a0-f9b64943b40e admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:28.323794 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 815/1605] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:28.324837 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369e2b4-6c22-4786-af10-c07106174246 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:28.336081 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 791/1606] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:28.336081 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a1088-fc5d-4f0c-96a0-f9b64943b40e admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:28.341902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369e2b4-6c22-4786-af10-c07106174246 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:28.348811 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca64f79-4c0d-47e6-84c5-312be7b06a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:28.349586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca64f79-4c0d-47e6-84c5-312be7b06a45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:28.349898 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca64f79-4c0d-47e6-84c5-312be7b06a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:28.350237 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca64f79-4c0d-47e6-84c5-312be7b06a45 admin admin] PATH_INFO: `/v3/projects/e03f09713c2e45d9a261416a5dbd0883/users/10fcb673c54e484a88a0dabe7b8edb15/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:28.372830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca64f79-4c0d-47e6-84c5-312be7b06a45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e03f09713c2e45d9a261416a5dbd0883, user_id=10fcb673c54e484a88a0dabe7b8edb15, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:28.376371 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca64f79-4c0d-47e6-84c5-312be7b06a45 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:28.391112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 816/1607] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:08:28 2022] PUT /identity/v3/projects/e03f09713c2e45d9a261416a5dbd0883/users/10fcb673c54e484a88a0dabe7b8edb15/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:28.392731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc39057-ce71-4a86-b19b-c110b3587c1a None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:28.393910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc39057-ce71-4a86-b19b-c110b3587c1a None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:28.394427 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc39057-ce71-4a86-b19b-c110b3587c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:28.427522 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-fdc39057-ce71-4a86-b19b-c110b3587c1a None None] MFA Rules not processed for user `de7c42ad8d41425ba19af08fda44949f`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:28.429177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-fdc39057-ce71-4a86-b19b-c110b3587c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:28.444984 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a1088-fc5d-4f0c-96a0-f9b64943b40e admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:28.506579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 817/1608] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:28.506579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc39057-ce71-4a86-b19b-c110b3587c1a None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:28.509824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f823749-b018-426e-a25e-023dda90bed0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:28.510536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f823749-b018-426e-a25e-023dda90bed0 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:28.510819 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f823749-b018-426e-a25e-023dda90bed0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:28.511111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f823749-b018-426e-a25e-023dda90bed0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:28.588363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f823749-b018-426e-a25e-023dda90bed0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:28.589806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f823749-b018-426e-a25e-023dda90bed0 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:28.601836 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 792/1609] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:08:28.602987 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f823749-b018-426e-a25e-023dda90bed0 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:28.607546 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc39057-ce71-4a86-b19b-c110b3587c1a None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:28.613719 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f823749-b018-426e-a25e-023dda90bed0 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:28.614319 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472265f7-40e2-4fa5-946c-1c47738e445f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:28.615202 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472265f7-40e2-4fa5-946c-1c47738e445f admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:28.616984 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472265f7-40e2-4fa5-946c-1c47738e445f admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:28.617712 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472265f7-40e2-4fa5-946c-1c47738e445f admin admin] PATH_INFO: `/v3/projects/6fde620889f34b04bb7c9fcd5ad0a368/users/bf58157cbc8e49c5bd913c68ff859bfb/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:28.619328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29303e0-a920-4a58-842a-7ab057f3af89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:28.620046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29303e0-a920-4a58-842a-7ab057f3af89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:28.620333 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29303e0-a920-4a58-842a-7ab057f3af89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:28.620610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29303e0-a920-4a58-842a-7ab057f3af89 admin admin] PATH_INFO: `/v3/projects/3450e9ca29fb44e4808fa52de2f88470/users/e8b5ce2249a84c01ab57c03b656b2949/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:28.636579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472265f7-40e2-4fa5-946c-1c47738e445f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fde620889f34b04bb7c9fcd5ad0a368, user_id=bf58157cbc8e49c5bd913c68ff859bfb, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:28.638270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29303e0-a920-4a58-842a-7ab057f3af89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3450e9ca29fb44e4808fa52de2f88470, user_id=e8b5ce2249a84c01ab57c03b656b2949, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:28.639433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472265f7-40e2-4fa5-946c-1c47738e445f admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:28.640973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29303e0-a920-4a58-842a-7ab057f3af89 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:28.643748 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:28.652961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:28.658154 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 818/1610] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:08:28 2022] PUT /identity/v3/projects/6fde620889f34b04bb7c9fcd5ad0a368/users/bf58157cbc8e49c5bd913c68ff859bfb/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:28.662976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 793/1611] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:08:28 2022] PUT /identity/v3/projects/3450e9ca29fb44e4808fa52de2f88470/users/e8b5ce2249a84c01ab57c03b656b2949/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:28.662976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d9351-d182-49b7-b1fb-f7fa991cd893 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:28.662976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d9351-d182-49b7-b1fb-f7fa991cd893 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:28.662976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d9351-d182-49b7-b1fb-f7fa991cd893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:28.663900 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf52a48-cf0c-4526-8477-d847a518b1bb None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:28.664633 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf52a48-cf0c-4526-8477-d847a518b1bb None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:28.664943 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf52a48-cf0c-4526-8477-d847a518b1bb None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:28.666850 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 819/1612] 10.222.0.36 () {54 vars in 962 bytes} [Fri Sep 2 09:08:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:08:28.667641 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf52a48-cf0c-4526-8477-d847a518b1bb None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:28.687970 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-b27d9351-d182-49b7-b1fb-f7fa991cd893 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:28.689826 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b27d9351-d182-49b7-b1fb-f7fa991cd893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:28.749963 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf52a48-cf0c-4526-8477-d847a518b1bb None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:28.754079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da204e0-980e-4ec8-8b12-f9104516b37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:28.754882 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da204e0-980e-4ec8-8b12-f9104516b37b admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:28.755425 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da204e0-980e-4ec8-8b12-f9104516b37b admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:28.755857 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da204e0-980e-4ec8-8b12-f9104516b37b admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:28.756721 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da204e0-980e-4ec8-8b12-f9104516b37b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:28.759100 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da204e0-980e-4ec8-8b12-f9104516b37b admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:28.770873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 794/1613] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:28.772243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92e07f-e88c-42c7-ab68-d619cedddcc4 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:28.772582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92e07f-e88c-42c7-ab68-d619cedddcc4 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:28.772909 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f92e07f-e88c-42c7-ab68-d619cedddcc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:28.796194 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-0f92e07f-e88c-42c7-ab68-d619cedddcc4 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:28.797581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-0f92e07f-e88c-42c7-ab68-d619cedddcc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:28.797722 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 820/1614] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:08:28 2022] POST /identity/v3/projects => generated 381 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:28.799225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046a425-d4f3-4a62-b879-bb26a9fb535e None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:28.799382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046a425-d4f3-4a62-b879-bb26a9fb535e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:28.799713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046a425-d4f3-4a62-b879-bb26a9fb535e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:28.821215 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-8046a425-d4f3-4a62-b879-bb26a9fb535e None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:28.822709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8046a425-d4f3-4a62-b879-bb26a9fb535e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:28.864178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 795/1615] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:28.864178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0fb03-3f60-470d-a78a-71be70cfbf9d None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:28.864178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0fb03-3f60-470d-a78a-71be70cfbf9d None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:28.864178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0fb03-3f60-470d-a78a-71be70cfbf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:28.901071 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 821/1616] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:28.902241 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046a425-d4f3-4a62-b879-bb26a9fb535e None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:28.918032 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-4da0fb03-3f60-470d-a78a-71be70cfbf9d None None] MFA Rules not processed for user `cce502a83f5343558c2136c0193e7d85`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:28.919367 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-4da0fb03-3f60-470d-a78a-71be70cfbf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:28.970088 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046a425-d4f3-4a62-b879-bb26a9fb535e None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:28.973992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f9c74-9ab4-42ad-b9fd-2d7248e47d0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:28.975211 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f9c74-9ab4-42ad-b9fd-2d7248e47d0f service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:28.975493 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f9c74-9ab4-42ad-b9fd-2d7248e47d0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:28.975767 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f9c74-9ab4-42ad-b9fd-2d7248e47d0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:28.986527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f9c74-9ab4-42ad-b9fd-2d7248e47d0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:28.988108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9f9c74-9ab4-42ad-b9fd-2d7248e47d0f service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:28.996038 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 796/1617] 10.222.0.36 () {58 vars in 1059 bytes} [Fri Sep 2 09:08:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:28.999426 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0fb03-3f60-470d-a78a-71be70cfbf9d None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:29.001965 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 822/1618] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:08:29.002950 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f9c74-9ab4-42ad-b9fd-2d7248e47d0f service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:29.012786 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f9c74-9ab4-42ad-b9fd-2d7248e47d0f service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:29.015265 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0fb03-3f60-470d-a78a-71be70cfbf9d None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:29.018457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b379d-14c0-436a-81bc-03cd366f66c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:29.019347 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b379d-14c0-436a-81bc-03cd366f66c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:29.019929 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b379d-14c0-436a-81bc-03cd366f66c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:29.020375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b379d-14c0-436a-81bc-03cd366f66c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:29.022029 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8805d-80c1-44b5-b9be-1b5f5c532668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:29.022647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8805d-80c1-44b5-b9be-1b5f5c532668 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:29.022927 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8805d-80c1-44b5-b9be-1b5f5c532668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:29.023234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8805d-80c1-44b5-b9be-1b5f5c532668 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:29.023872 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8805d-80c1-44b5-b9be-1b5f5c532668 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:29.026280 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8805d-80c1-44b5-b9be-1b5f5c532668 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:29.035015 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b379d-14c0-436a-81bc-03cd366f66c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:29.037456 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b379d-14c0-436a-81bc-03cd366f66c7 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:29.050720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 823/1619] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:08:29.052437 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b379d-14c0-436a-81bc-03cd366f66c7 service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:29.062217 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 797/1620] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:08:28 2022] POST /identity/v3/users => generated 297 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:29.064991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8805d-80c1-44b5-b9be-1b5f5c532668 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:29.083323 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b379d-14c0-436a-81bc-03cd366f66c7 service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:29.087212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a51d0-76b6-4cfc-9838-aa5322640a4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:29.087894 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a51d0-76b6-4cfc-9838-aa5322640a4f service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:29.088290 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a51d0-76b6-4cfc-9838-aa5322640a4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:29.088630 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a51d0-76b6-4cfc-9838-aa5322640a4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:29.091174 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:29.100788 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a51d0-76b6-4cfc-9838-aa5322640a4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:29.102088 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8805d-80c1-44b5-b9be-1b5f5c532668 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:29.102654 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a51d0-76b6-4cfc-9838-aa5322640a4f service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:29.108436 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:29.119396 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 824/1621] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:08:29.121211 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a51d0-76b6-4cfc-9838-aa5322640a4f service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:29.127649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced9d20c-aa9c-4acf-93ec-c3b7a995993d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:29.128491 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9d20c-aa9c-4acf-93ec-c3b7a995993d service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:29.128802 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9d20c-aa9c-4acf-93ec-c3b7a995993d service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:29.129047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced9d20c-aa9c-4acf-93ec-c3b7a995993d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:29.138425 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a51d0-76b6-4cfc-9838-aa5322640a4f service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:29.145721 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55637bac-0dd3-4d8c-90a4-79d80354e4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:29.146485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55637bac-0dd3-4d8c-90a4-79d80354e4e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:29.146823 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55637bac-0dd3-4d8c-90a4-79d80354e4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:29.147408 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55637bac-0dd3-4d8c-90a4-79d80354e4e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:29.148567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55637bac-0dd3-4d8c-90a4-79d80354e4e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:29.150687 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55637bac-0dd3-4d8c-90a4-79d80354e4e6 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:29.167722 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 825/1622] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:29.172829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55637bac-0dd3-4d8c-90a4-79d80354e4e6 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:29.184645 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55637bac-0dd3-4d8c-90a4-79d80354e4e6 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:29.194158 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5e832-0b90-4d12-90c4-d88701f72bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:29.195724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5e832-0b90-4d12-90c4-d88701f72bb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:29.196084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5e832-0b90-4d12-90c4-d88701f72bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:29.196597 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5e832-0b90-4d12-90c4-d88701f72bb6 admin admin] PATH_INFO: `/v3/projects/d6241750f5464148a8e622734b4034e2/users/421e3f1efeb24faeaf89cb15822d002f/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:29.216014 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c5e832-0b90-4d12-90c4-d88701f72bb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6241750f5464148a8e622734b4034e2, user_id=421e3f1efeb24faeaf89cb15822d002f, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:29.219817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c5e832-0b90-4d12-90c4-d88701f72bb6 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:29.241388 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 826/1623] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:08:29 2022] PUT /identity/v3/projects/d6241750f5464148a8e622734b4034e2/users/421e3f1efeb24faeaf89cb15822d002f/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:29.243529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5e832-0b90-4d12-90c4-d88701f72bb6 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:29.279920 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced9d20c-aa9c-4acf-93ec-c3b7a995993d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:29.285514 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced9d20c-aa9c-4acf-93ec-c3b7a995993d service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:29.311154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 798/1624] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:08:29.341121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5e832-0b90-4d12-90c4-d88701f72bb6 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:29.347504 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d936f5-0fd8-493b-95a0-1315d52420a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:29.348117 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d936f5-0fd8-493b-95a0-1315d52420a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:29.348437 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d936f5-0fd8-493b-95a0-1315d52420a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:29.348767 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d936f5-0fd8-493b-95a0-1315d52420a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:29.349754 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d936f5-0fd8-493b-95a0-1315d52420a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:29.352807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d936f5-0fd8-493b-95a0-1315d52420a6 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:29.376890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 827/1625] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:29.378243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced9d20c-aa9c-4acf-93ec-c3b7a995993d service nova] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:29.394415 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced9d20c-aa9c-4acf-93ec-c3b7a995993d service nova] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:29.406819 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3584ff-8786-4bd0-810c-a9c764fb7980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:29.407737 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3584ff-8786-4bd0-810c-a9c764fb7980 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:29.408022 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3584ff-8786-4bd0-810c-a9c764fb7980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:29.408447 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3584ff-8786-4bd0-810c-a9c764fb7980 admin admin] PATH_INFO: `/v3/projects/d6241750f5464148a8e622734b4034e2/users/421e3f1efeb24faeaf89cb15822d002f/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:29.410999 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3584ff-8786-4bd0-810c-a9c764fb7980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6241750f5464148a8e622734b4034e2, user_id=421e3f1efeb24faeaf89cb15822d002f, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:29.413718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3584ff-8786-4bd0-810c-a9c764fb7980 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:29.443941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 799/1626] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:08:29 2022] PUT /identity/v3/projects/d6241750f5464148a8e622734b4034e2/users/421e3f1efeb24faeaf89cb15822d002f/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:29.454932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f4ce7-bb09-4584-8887-3f7c3fd68b42 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:29.455303 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f4ce7-bb09-4584-8887-3f7c3fd68b42 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:29.455543 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f4ce7-bb09-4584-8887-3f7c3fd68b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:29.518406 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-7d3f4ce7-bb09-4584-8887-3f7c3fd68b42 None None] MFA Rules not processed for user `421e3f1efeb24faeaf89cb15822d002f`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:29.520413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-7d3f4ce7-bb09-4584-8887-3f7c3fd68b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:29.677864 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 828/1627] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:29.685642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3584ff-8786-4bd0-810c-a9c764fb7980 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:29.798968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3584ff-8786-4bd0-810c-a9c764fb7980 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:29.808011 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280f21c-e08c-49ae-ae2f-76ba891d0de6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:29.808804 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280f21c-e08c-49ae-ae2f-76ba891d0de6 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:29.808923 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280f21c-e08c-49ae-ae2f-76ba891d0de6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:29.809200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280f21c-e08c-49ae-ae2f-76ba891d0de6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:29.811895 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6ea04-7d00-4070-bcc6-ae5e5f5874d0 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:29.812178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6ea04-7d00-4070-bcc6-ae5e5f5874d0 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:29.812454 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a6ea04-7d00-4070-bcc6-ae5e5f5874d0 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:29.813843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 829/1628] 10.222.0.36 () {54 vars in 962 bytes} [Fri Sep 2 09:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:08:29.821968 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd485b2a-86e9-4575-ba54-d4306ac6eedd None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:29.822253 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd485b2a-86e9-4575-ba54-d4306ac6eedd None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:29.822526 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd485b2a-86e9-4575-ba54-d4306ac6eedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:29.857260 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280f21c-e08c-49ae-ae2f-76ba891d0de6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:29.858744 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e280f21c-e08c-49ae-ae2f-76ba891d0de6 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:29.868177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-fd485b2a-86e9-4575-ba54-d4306ac6eedd None None] MFA Rules not processed for user `cce502a83f5343558c2136c0193e7d85`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:29.869440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-fd485b2a-86e9-4575-ba54-d4306ac6eedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:29.885925 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 800/1629] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:08:29.890376 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc15a4-8606-4a8b-bf5e-aaaf8b3fe0d0 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:29.890666 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc15a4-8606-4a8b-bf5e-aaaf8b3fe0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:29.890955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc15a4-8606-4a8b-bf5e-aaaf8b3fe0d0 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:29.894187 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 801/1630] 10.222.0.36 () {54 vars in 912 bytes} [Fri Sep 2 09:08:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:08:29.908578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177702a3-34c0-42ad-b35d-8a1d93845934 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:29.908882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177702a3-34c0-42ad-b35d-8a1d93845934 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:29.909244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177702a3-34c0-42ad-b35d-8a1d93845934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:29.949661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-177702a3-34c0-42ad-b35d-8a1d93845934 None None] MFA Rules not processed for user `cce502a83f5343558c2136c0193e7d85`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:29.950950 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-177702a3-34c0-42ad-b35d-8a1d93845934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:29.971673 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:30.044104 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 830/1631] 10.222.0.36 () {58 vars in 1059 bytes} [Fri Sep 2 09:08:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:08:30.050502 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86921f1c-d9d2-4606-8b81-0043237808fb None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:30.050789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86921f1c-d9d2-4606-8b81-0043237808fb None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:30.051064 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86921f1c-d9d2-4606-8b81-0043237808fb None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:30.053658 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 831/1632] 10.222.0.36 () {54 vars in 912 bytes} [Fri Sep 2 09:08:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:08:30.061301 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6f074-bce8-474a-b069-be2fa75cb2a4 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:30.062321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6f074-bce8-474a-b069-be2fa75cb2a4 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:30.062321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6f074-bce8-474a-b069-be2fa75cb2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:30.094179 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-5ad6f074-bce8-474a-b069-be2fa75cb2a4 None None] MFA Rules not processed for user `cce502a83f5343558c2136c0193e7d85`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:30.097009 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5ad6f074-bce8-474a-b069-be2fa75cb2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:30.111345 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 802/1633] 10.222.0.36 () {58 vars in 1009 bytes} [Fri Sep 2 09:08:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 02 09:08:30.119686 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177702a3-34c0-42ad-b35d-8a1d93845934 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:30.120182 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 832/1634] 10.222.0.36 () {58 vars in 1009 bytes} [Fri Sep 2 09:08:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:30.134121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177702a3-34c0-42ad-b35d-8a1d93845934 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:30.139124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0a362-7a38-4b98-8cae-b772a3676706 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:30.139124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0a362-7a38-4b98-8cae-b772a3676706 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:30.139124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0a362-7a38-4b98-8cae-b772a3676706 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:30.139124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0a362-7a38-4b98-8cae-b772a3676706 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:30.151823 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0a362-7a38-4b98-8cae-b772a3676706 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:30.153277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0a362-7a38-4b98-8cae-b772a3676706 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:30.156100 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6f074-bce8-474a-b069-be2fa75cb2a4 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:30.168584 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6f074-bce8-474a-b069-be2fa75cb2a4 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:30.178160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b421a747-b6df-4669-b5c8-9707fa5f6c6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:30.178856 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421a747-b6df-4669-b5c8-9707fa5f6c6d service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:30.179166 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421a747-b6df-4669-b5c8-9707fa5f6c6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:30.179606 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b421a747-b6df-4669-b5c8-9707fa5f6c6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:30.199976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 803/1635] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:08:30.214470 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421a747-b6df-4669-b5c8-9707fa5f6c6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:30.215789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b421a747-b6df-4669-b5c8-9707fa5f6c6d service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:30.245036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 833/1636] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:08:31.995322 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0a362-7a38-4b98-8cae-b772a3676706 service cinder] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:32.017311 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:32.093183 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0a362-7a38-4b98-8cae-b772a3676706 service cinder] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:32.108698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641078dd-aec6-4ef5-9ccb-8a8b91b8db3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:32.110422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641078dd-aec6-4ef5-9ccb-8a8b91b8db3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:32.111899 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641078dd-aec6-4ef5-9ccb-8a8b91b8db3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:32.112492 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641078dd-aec6-4ef5-9ccb-8a8b91b8db3f admin admin] PATH_INFO: `/v3/users/de7c42ad8d41425ba19af08fda44949f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:32.114303 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641078dd-aec6-4ef5-9ccb-8a8b91b8db3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=de7c42ad8d41425ba19af08fda44949f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:32.117474 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641078dd-aec6-4ef5-9ccb-8a8b91b8db3f admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:32.222036 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-641078dd-aec6-4ef5-9ccb-8a8b91b8db3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de7c42ad8d41425ba19af08fda44949f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:32.224108 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-641078dd-aec6-4ef5-9ccb-8a8b91b8db3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de7c42ad8d41425ba19af08fda44949f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:32.256142 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-641078dd-aec6-4ef5-9ccb-8a8b91b8db3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de7c42ad8d41425ba19af08fda44949f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:32.267289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-641078dd-aec6-4ef5-9ccb-8a8b91b8db3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de7c42ad8d41425ba19af08fda44949f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:32.271852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-641078dd-aec6-4ef5-9ccb-8a8b91b8db3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de7c42ad8d41425ba19af08fda44949f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:32.295148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 804/1637] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:08:31 2022] DELETE /identity/v3/users/de7c42ad8d41425ba19af08fda44949f => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:08:32.299663 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fc88c-fe4d-44c3-9212-ad676268427f None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:32.300221 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fc88c-fe4d-44c3-9212-ad676268427f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:32.300452 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58fc88c-fe4d-44c3-9212-ad676268427f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:32.362837 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-b58fc88c-fe4d-44c3-9212-ad676268427f None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:32.364213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-b58fc88c-fe4d-44c3-9212-ad676268427f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:32.531920 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 834/1638] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:32.588005 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641078dd-aec6-4ef5-9ccb-8a8b91b8db3f admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:32.589674 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-641078dd-aec6-4ef5-9ccb-8a8b91b8db3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:32.703724 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641078dd-aec6-4ef5-9ccb-8a8b91b8db3f admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:32.714367 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59b2f4-9b5c-47de-989c-3841fa219d18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:32.723063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59b2f4-9b5c-47de-989c-3841fa219d18 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:32.725246 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59b2f4-9b5c-47de-989c-3841fa219d18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:32.732281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59b2f4-9b5c-47de-989c-3841fa219d18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:32.762458 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59b2f4-9b5c-47de-989c-3841fa219d18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:32.763958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59b2f4-9b5c-47de-989c-3841fa219d18 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:32.788261 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 805/1639] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:08:33.945228 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58fc88c-fe4d-44c3-9212-ad676268427f None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:33.947019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-b58fc88c-fe4d-44c3-9212-ad676268427f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:33.972474 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58fc88c-fe4d-44c3-9212-ad676268427f None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:33.980067 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587937c7-3be0-4edc-8bcd-007b3292a7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:33.980954 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587937c7-3be0-4edc-8bcd-007b3292a7d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:33.981400 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587937c7-3be0-4edc-8bcd-007b3292a7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:33.981860 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587937c7-3be0-4edc-8bcd-007b3292a7d7 admin admin] PATH_INFO: `/v3/users/24be3210cc564b869eaa76567ff83c63` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:33.983523 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587937c7-3be0-4edc-8bcd-007b3292a7d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24be3210cc564b869eaa76567ff83c63)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:33.984761 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: WARNING py.warnings [None req-587937c7-3be0-4edc-8bcd-007b3292a7d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:08:33.984761 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: warnings.warn(msg) Sep 02 09:08:33.984761 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: Sep 02 09:08:33.989370 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587937c7-3be0-4edc-8bcd-007b3292a7d7 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:34.121540 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59b2f4-9b5c-47de-989c-3841fa219d18 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:34.124790 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6f59b2f4-9b5c-47de-989c-3841fa219d18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:34.151406 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59b2f4-9b5c-47de-989c-3841fa219d18 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:34.179355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557d533-3d40-435b-aaf4-0486321c0d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:34.179355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557d533-3d40-435b-aaf4-0486321c0d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:34.180241 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557d533-3d40-435b-aaf4-0486321c0d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:34.180995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557d533-3d40-435b-aaf4-0486321c0d43 admin admin] PATH_INFO: `/v3/projects/ced6a1e5137e41cf9f3c5935b2c65ae8` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:34.181261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-587937c7-3be0-4edc-8bcd-007b3292a7d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24be3210cc564b869eaa76567ff83c63'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:34.186373 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557d533-3d40-435b-aaf4-0486321c0d43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ced6a1e5137e41cf9f3c5935b2c65ae8)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:34.191491 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: WARNING py.warnings [None req-f557d533-3d40-435b-aaf4-0486321c0d43 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:08:34.191491 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: warnings.warn(msg) Sep 02 09:08:34.191491 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: Sep 02 09:08:34.195352 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557d533-3d40-435b-aaf4-0486321c0d43 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:34.211385 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-587937c7-3be0-4edc-8bcd-007b3292a7d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24be3210cc564b869eaa76567ff83c63'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:34.225425 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-587937c7-3be0-4edc-8bcd-007b3292a7d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24be3210cc564b869eaa76567ff83c63'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:34.226062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-587937c7-3be0-4edc-8bcd-007b3292a7d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24be3210cc564b869eaa76567ff83c63'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:34.226703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-587937c7-3be0-4edc-8bcd-007b3292a7d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24be3210cc564b869eaa76567ff83c63'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:34.286994 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:34.296846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 835/1640] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:08:33 2022] DELETE /identity/v3/users/24be3210cc564b869eaa76567ff83c63 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:34.301209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553da16-07eb-48fe-a6b6-721cb09226a4 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:34.302281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553da16-07eb-48fe-a6b6-721cb09226a4 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:34.302281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553da16-07eb-48fe-a6b6-721cb09226a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:34.335025 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-6553da16-07eb-48fe-a6b6-721cb09226a4 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:34.337235 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6553da16-07eb-48fe-a6b6-721cb09226a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:34.349724 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f557d533-3d40-435b-aaf4-0486321c0d43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ced6a1e5137e41cf9f3c5935b2c65ae8', 'request_id': 'req-f557d533-3d40-435b-aaf4-0486321c0d43'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:34.379531 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:34.387556 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f557d533-3d40-435b-aaf4-0486321c0d43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ced6a1e5137e41cf9f3c5935b2c65ae8', 'request_id': 'req-f557d533-3d40-435b-aaf4-0486321c0d43'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:34.409688 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f557d533-3d40-435b-aaf4-0486321c0d43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ced6a1e5137e41cf9f3c5935b2c65ae8', 'request_id': 'req-f557d533-3d40-435b-aaf4-0486321c0d43'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:34.441169 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f557d533-3d40-435b-aaf4-0486321c0d43 admin admin] The token cache is being invalidate because project ced6a1e5137e41cf9f3c5935b2c65ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:08:34.441502 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f557d533-3d40-435b-aaf4-0486321c0d43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:34.442250 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f557d533-3d40-435b-aaf4-0486321c0d43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:34.446823 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 806/1641] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:08:34 2022] DELETE /identity/v3/projects/ced6a1e5137e41cf9f3c5935b2c65ae8 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:34.451414 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557d533-3d40-435b-aaf4-0486321c0d43 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:34.455764 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-f557d533-3d40-435b-aaf4-0486321c0d43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:34.475563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 836/1642] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:08:34.487897 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553da16-07eb-48fe-a6b6-721cb09226a4 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:34.489509 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6553da16-07eb-48fe-a6b6-721cb09226a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:34.649194 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557d533-3d40-435b-aaf4-0486321c0d43 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:34.650433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553da16-07eb-48fe-a6b6-721cb09226a4 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:34.665680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc25ce9-60e8-476c-ab1a-e1b173eec2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:34.666529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a36c9-1f64-4969-86a4-3e1a995eedd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:34.667307 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a36c9-1f64-4969-86a4-3e1a995eedd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:34.670961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc25ce9-60e8-476c-ab1a-e1b173eec2de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:34.671749 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc25ce9-60e8-476c-ab1a-e1b173eec2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:34.671749 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc25ce9-60e8-476c-ab1a-e1b173eec2de admin admin] PATH_INFO: `/v3/users/421e3f1efeb24faeaf89cb15822d002f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:34.671922 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a36c9-1f64-4969-86a4-3e1a995eedd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:34.674058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a36c9-1f64-4969-86a4-3e1a995eedd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:34.676774 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc25ce9-60e8-476c-ab1a-e1b173eec2de admin admin] RBAC: Authorizing `identity:delete_user(user_id=421e3f1efeb24faeaf89cb15822d002f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:34.679553 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc25ce9-60e8-476c-ab1a-e1b173eec2de admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:34.698028 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a36c9-1f64-4969-86a4-3e1a995eedd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:34.703729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a36c9-1f64-4969-86a4-3e1a995eedd8 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:34.762633 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 837/1643] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:08:34.814741 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7bc25ce9-60e8-476c-ab1a-e1b173eec2de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '421e3f1efeb24faeaf89cb15822d002f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:34.818563 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7bc25ce9-60e8-476c-ab1a-e1b173eec2de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '421e3f1efeb24faeaf89cb15822d002f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:34.849824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7bc25ce9-60e8-476c-ab1a-e1b173eec2de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '421e3f1efeb24faeaf89cb15822d002f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:34.870242 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7bc25ce9-60e8-476c-ab1a-e1b173eec2de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '421e3f1efeb24faeaf89cb15822d002f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:34.875717 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7bc25ce9-60e8-476c-ab1a-e1b173eec2de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '421e3f1efeb24faeaf89cb15822d002f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:34.914256 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 807/1644] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:08:34 2022] DELETE /identity/v3/users/421e3f1efeb24faeaf89cb15822d002f => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:36.683924 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a36c9-1f64-4969-86a4-3e1a995eedd8 service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:36.695112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a36c9-1f64-4969-86a4-3e1a995eedd8 service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:36.702294 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc25ce9-60e8-476c-ab1a-e1b173eec2de admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:36.703891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7bc25ce9-60e8-476c-ab1a-e1b173eec2de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:36.804711 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:36.886223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc25ce9-60e8-476c-ab1a-e1b173eec2de admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:36.888053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa79d189-b928-429f-9f18-38e5f460de2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:36.888707 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79d189-b928-429f-9f18-38e5f460de2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:36.888989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79d189-b928-429f-9f18-38e5f460de2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:36.889269 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa79d189-b928-429f-9f18-38e5f460de2d admin admin] PATH_INFO: `/v3/projects/d6241750f5464148a8e622734b4034e2` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:36.891639 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79d189-b928-429f-9f18-38e5f460de2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6241750f5464148a8e622734b4034e2)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:36.894472 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa79d189-b928-429f-9f18-38e5f460de2d admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:36.901705 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f6bf5-08d7-457a-a5a9-e27263b12c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:36.904204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f6bf5-08d7-457a-a5a9-e27263b12c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:36.904751 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f6bf5-08d7-457a-a5a9-e27263b12c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:36.906060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f6bf5-08d7-457a-a5a9-e27263b12c97 admin admin] PATH_INFO: `/v3/projects/6499e3f183d4412783693d05bd96c76d` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:36.912849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f6bf5-08d7-457a-a5a9-e27263b12c97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6499e3f183d4412783693d05bd96c76d)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:36.918942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f6bf5-08d7-457a-a5a9-e27263b12c97 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:36.978605 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-fa79d189-b928-429f-9f18-38e5f460de2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6241750f5464148a8e622734b4034e2', 'request_id': 'req-fa79d189-b928-429f-9f18-38e5f460de2d'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:36.996835 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-fa79d189-b928-429f-9f18-38e5f460de2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6241750f5464148a8e622734b4034e2', 'request_id': 'req-fa79d189-b928-429f-9f18-38e5f460de2d'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:37.009554 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-fa79d189-b928-429f-9f18-38e5f460de2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6241750f5464148a8e622734b4034e2', 'request_id': 'req-fa79d189-b928-429f-9f18-38e5f460de2d'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:37.036559 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c99f6bf5-08d7-457a-a5a9-e27263b12c97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6499e3f183d4412783693d05bd96c76d', 'request_id': 'req-c99f6bf5-08d7-457a-a5a9-e27263b12c97'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:37.051272 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-fa79d189-b928-429f-9f18-38e5f460de2d admin admin] The token cache is being invalidate because project d6241750f5464148a8e622734b4034e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:08:37.051648 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-fa79d189-b928-429f-9f18-38e5f460de2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:37.053159 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-fa79d189-b928-429f-9f18-38e5f460de2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:37.056542 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 838/1645] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:08:36 2022] DELETE /identity/v3/projects/d6241750f5464148a8e622734b4034e2 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:37.057867 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c99f6bf5-08d7-457a-a5a9-e27263b12c97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6499e3f183d4412783693d05bd96c76d', 'request_id': 'req-c99f6bf5-08d7-457a-a5a9-e27263b12c97'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:37.062375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c03db1-03e1-48c3-93f8-6b2e39c6791c None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:37.062693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c03db1-03e1-48c3-93f8-6b2e39c6791c None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:37.062946 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c03db1-03e1-48c3-93f8-6b2e39c6791c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:37.074196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c99f6bf5-08d7-457a-a5a9-e27263b12c97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6499e3f183d4412783693d05bd96c76d', 'request_id': 'req-c99f6bf5-08d7-457a-a5a9-e27263b12c97'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:37.090840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:37.105606 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c99f6bf5-08d7-457a-a5a9-e27263b12c97 admin admin] The token cache is being invalidate because project 6499e3f183d4412783693d05bd96c76d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:08:37.106142 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c99f6bf5-08d7-457a-a5a9-e27263b12c97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:37.107309 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c99f6bf5-08d7-457a-a5a9-e27263b12c97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:37.113302 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 808/1646] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:08:36 2022] DELETE /identity/v3/projects/6499e3f183d4412783693d05bd96c76d => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:37.118192 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89351669-a325-4ccb-a4d7-8160800a76e9 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:37.118728 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89351669-a325-4ccb-a4d7-8160800a76e9 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:37.119115 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89351669-a325-4ccb-a4d7-8160800a76e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:37.129286 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-e1c03db1-03e1-48c3-93f8-6b2e39c6791c None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:37.130817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-e1c03db1-03e1-48c3-93f8-6b2e39c6791c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:37.187253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-89351669-a325-4ccb-a4d7-8160800a76e9 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:37.190377 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-89351669-a325-4ccb-a4d7-8160800a76e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:37.333928 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 839/1647] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:37.342655 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03c441-c684-471d-a948-84a3c3c9a93f None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:37.342963 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03c441-c684-471d-a948-84a3c3c9a93f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:37.343447 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03c441-c684-471d-a948-84a3c3c9a93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:37.390585 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 809/1648] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:08:37.398915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0afcd2-4a8e-4322-8755-5ab3edec0bcf None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:37.399269 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0afcd2-4a8e-4322-8755-5ab3edec0bcf None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:37.399603 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0afcd2-4a8e-4322-8755-5ab3edec0bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:37.403353 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-0f03c441-c684-471d-a948-84a3c3c9a93f None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:37.406569 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0f03c441-c684-471d-a948-84a3c3c9a93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:37.433206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 840/1649] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:08:37.434329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03c441-c684-471d-a948-84a3c3c9a93f None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:37.448724 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-2f0afcd2-4a8e-4322-8755-5ab3edec0bcf None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:37.450050 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-2f0afcd2-4a8e-4322-8755-5ab3edec0bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:37.453279 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03c441-c684-471d-a948-84a3c3c9a93f None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:37.461123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6125e-d925-49d9-b798-daed10516776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:37.461747 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6125e-d925-49d9-b798-daed10516776 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:37.462031 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6125e-d925-49d9-b798-daed10516776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:37.462302 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6125e-d925-49d9-b798-daed10516776 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:37.462959 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6125e-d925-49d9-b798-daed10516776 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:08:37.463296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6125e-d925-49d9-b798-daed10516776 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:37.465337 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b6125e-d925-49d9-b798-daed10516776 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:37.479480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 810/1650] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:37.483496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0afcd2-4a8e-4322-8755-5ab3edec0bcf None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:37.486741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 841/1651] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:08:37.487708 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6125e-d925-49d9-b798-daed10516776 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:37.495019 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0afcd2-4a8e-4322-8755-5ab3edec0bcf None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:37.497931 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6125e-d925-49d9-b798-daed10516776 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:37.501006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a21c2-3f06-41cb-935a-f3db99b6b329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:37.502243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a21c2-3f06-41cb-935a-f3db99b6b329 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:37.502845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a21c2-3f06-41cb-935a-f3db99b6b329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:37.502966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a21c2-3f06-41cb-935a-f3db99b6b329 admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:37.503486 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0227ced-1547-414b-b67c-d0676506a473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:37.503834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a21c2-3f06-41cb-935a-f3db99b6b329 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:08:37.504213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0227ced-1547-414b-b67c-d0676506a473 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:37.504339 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a21c2-3f06-41cb-935a-f3db99b6b329 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:37.504602 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0227ced-1547-414b-b67c-d0676506a473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:37.504941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0227ced-1547-414b-b67c-d0676506a473 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:37.505639 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0227ced-1547-414b-b67c-d0676506a473 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:37.506543 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a21c2-3f06-41cb-935a-f3db99b6b329 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:37.507796 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0227ced-1547-414b-b67c-d0676506a473 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:37.522327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 811/1652] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:08:37.528134 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a21c2-3f06-41cb-935a-f3db99b6b329 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:37.547800 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a21c2-3f06-41cb-935a-f3db99b6b329 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:37.549265 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 842/1653] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:08:37 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:37.553608 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0227ced-1547-414b-b67c-d0676506a473 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:37.644442 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0227ced-1547-414b-b67c-d0676506a473 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:37.657955 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28446ecc-b57d-4b83-b228-498423d3070a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:37.660744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28446ecc-b57d-4b83-b228-498423d3070a admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:37.661216 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28446ecc-b57d-4b83-b228-498423d3070a admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:37.663526 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28446ecc-b57d-4b83-b228-498423d3070a admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:37.664516 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28446ecc-b57d-4b83-b228-498423d3070a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:37.669623 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28446ecc-b57d-4b83-b228-498423d3070a admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:37.675328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:37.710634 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136519e0-4f91-4f55-bc64-5443d6c95c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:37.712306 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136519e0-4f91-4f55-bc64-5443d6c95c3e admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:37.712892 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136519e0-4f91-4f55-bc64-5443d6c95c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:37.713303 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136519e0-4f91-4f55-bc64-5443d6c95c3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:37.715096 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136519e0-4f91-4f55-bc64-5443d6c95c3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:37.718440 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136519e0-4f91-4f55-bc64-5443d6c95c3e admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:37.722935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 843/1654] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:08:37 2022] POST /identity/v3/users => generated 292 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:37.728039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28446ecc-b57d-4b83-b228-498423d3070a admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:37.739943 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28446ecc-b57d-4b83-b228-498423d3070a admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:37.764610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d65b84-3f42-4e56-8f6e-8b70c06e67e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:37.765231 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d65b84-3f42-4e56-8f6e-8b70c06e67e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:37.765519 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d65b84-3f42-4e56-8f6e-8b70c06e67e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:37.765804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d65b84-3f42-4e56-8f6e-8b70c06e67e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:37.766551 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d65b84-3f42-4e56-8f6e-8b70c06e67e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:37.768253 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d65b84-3f42-4e56-8f6e-8b70c06e67e4 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:37.769140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 812/1655] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:08:37 2022] POST /identity/v3/projects => generated 405 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:37.773176 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136519e0-4f91-4f55-bc64-5443d6c95c3e admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:37.788414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 844/1656] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:37.799106 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d65b84-3f42-4e56-8f6e-8b70c06e67e4 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:37.809285 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d65b84-3f42-4e56-8f6e-8b70c06e67e4 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:37.867343 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136519e0-4f91-4f55-bc64-5443d6c95c3e admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:37.875052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f95c66-7b92-49e7-957f-363f982b196d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:37.876598 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f95c66-7b92-49e7-957f-363f982b196d admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:37.877086 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f95c66-7b92-49e7-957f-363f982b196d admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:37.877658 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f95c66-7b92-49e7-957f-363f982b196d admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:37.879657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f95c66-7b92-49e7-957f-363f982b196d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:37.884332 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f95c66-7b92-49e7-957f-363f982b196d admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:37.902925 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:37.937321 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 813/1657] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:08:37 2022] POST /identity/v3/users => generated 303 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:37.939950 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f95c66-7b92-49e7-957f-363f982b196d admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:37.944633 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f039f4-25ae-4bb7-b79c-c736a9486658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:37.945349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f039f4-25ae-4bb7-b79c-c736a9486658 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:37.945889 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f039f4-25ae-4bb7-b79c-c736a9486658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:37.946166 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f039f4-25ae-4bb7-b79c-c736a9486658 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:37.946820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f039f4-25ae-4bb7-b79c-c736a9486658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:37.948478 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f039f4-25ae-4bb7-b79c-c736a9486658 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:37.957891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f95c66-7b92-49e7-957f-363f982b196d admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:37.968855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 845/1658] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:37.969140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2b1b7-cacb-4be0-842e-6a8d8a9c250a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:37.970675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2b1b7-cacb-4be0-842e-6a8d8a9c250a admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:37.971135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2b1b7-cacb-4be0-842e-6a8d8a9c250a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:37.971527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2b1b7-cacb-4be0-842e-6a8d8a9c250a admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:37.975128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2b1b7-cacb-4be0-842e-6a8d8a9c250a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:37.976165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f039f4-25ae-4bb7-b79c-c736a9486658 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:37.977952 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2b1b7-cacb-4be0-842e-6a8d8a9c250a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:37.990954 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 814/1659] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:37.994913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2b1b7-cacb-4be0-842e-6a8d8a9c250a admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:38.000819 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f039f4-25ae-4bb7-b79c-c736a9486658 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:38.015104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2b1b7-cacb-4be0-842e-6a8d8a9c250a admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:38.025590 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2af41-e420-42bb-8f53-bbb674c17ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:38.026410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2af41-e420-42bb-8f53-bbb674c17ada admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:38.026686 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2af41-e420-42bb-8f53-bbb674c17ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:38.026966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2af41-e420-42bb-8f53-bbb674c17ada admin admin] PATH_INFO: `/v3/projects/0bf099dbeecf43569b5e7ff0cdd6ac73/users/f4ffc9fc81a74eeba1ea05488fda1f41/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:38.028739 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1cc54-8118-441c-8d36-d6b96197af99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:38.029534 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1cc54-8118-441c-8d36-d6b96197af99 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:38.030022 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1cc54-8118-441c-8d36-d6b96197af99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:38.030437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1cc54-8118-441c-8d36-d6b96197af99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:38.032104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c1cc54-8118-441c-8d36-d6b96197af99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:38.034561 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c1cc54-8118-441c-8d36-d6b96197af99 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:38.050503 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 815/1660] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:37 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:38.056217 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1cc54-8118-441c-8d36-d6b96197af99 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:38.069568 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2af41-e420-42bb-8f53-bbb674c17ada admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bf099dbeecf43569b5e7ff0cdd6ac73, user_id=f4ffc9fc81a74eeba1ea05488fda1f41, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:38.072421 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1cc54-8118-441c-8d36-d6b96197af99 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:38.073159 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2af41-e420-42bb-8f53-bbb674c17ada admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:38.085504 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865d42e-0872-43d9-b661-dedcd3f046b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:38.087453 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865d42e-0872-43d9-b661-dedcd3f046b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:38.087929 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865d42e-0872-43d9-b661-dedcd3f046b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:38.088353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865d42e-0872-43d9-b661-dedcd3f046b2 admin admin] PATH_INFO: `/v3/projects/d405b5c6dec74ddd85a379135a549d7b/users/a6e4c04db8af40f288e2b86afb149ec4/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:38.100934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 846/1661] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:08:37 2022] PUT /identity/v3/projects/0bf099dbeecf43569b5e7ff0cdd6ac73/users/f4ffc9fc81a74eeba1ea05488fda1f41/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:38.116233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ebfb9-0b1f-4716-9fe3-9fdda0877650 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:38.116528 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ebfb9-0b1f-4716-9fe3-9fdda0877650 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:38.116825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ebfb9-0b1f-4716-9fe3-9fdda0877650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:38.127235 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865d42e-0872-43d9-b661-dedcd3f046b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d405b5c6dec74ddd85a379135a549d7b, user_id=a6e4c04db8af40f288e2b86afb149ec4, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:38.129980 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865d42e-0872-43d9-b661-dedcd3f046b2 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:38.153797 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-928ebfb9-0b1f-4716-9fe3-9fdda0877650 None None] MFA Rules not processed for user `f4ffc9fc81a74eeba1ea05488fda1f41`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:38.156848 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-928ebfb9-0b1f-4716-9fe3-9fdda0877650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:38.166405 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 816/1662] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:08:38 2022] PUT /identity/v3/projects/d405b5c6dec74ddd85a379135a549d7b/users/a6e4c04db8af40f288e2b86afb149ec4/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:38.168470 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31a63d-6b87-4947-8697-dded704324c3 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:38.168756 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31a63d-6b87-4947-8697-dded704324c3 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:38.169027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b31a63d-6b87-4947-8697-dded704324c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:38.212010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-6b31a63d-6b87-4947-8697-dded704324c3 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:38.213244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6b31a63d-6b87-4947-8697-dded704324c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:38.309926 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 847/1663] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:38.312582 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ebfb9-0b1f-4716-9fe3-9fdda0877650 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:38.339993 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 817/1664] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:38.519392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ebfb9-0b1f-4716-9fe3-9fdda0877650 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:38.529843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6ca89-35c6-4565-8a2d-154507add00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:38.530486 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6ca89-35c6-4565-8a2d-154507add00a admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:38.531401 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6ca89-35c6-4565-8a2d-154507add00a admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:38.531693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6ca89-35c6-4565-8a2d-154507add00a admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:38.532385 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6ca89-35c6-4565-8a2d-154507add00a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:38.534354 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6ca89-35c6-4565-8a2d-154507add00a admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:38.583482 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31a63d-6b87-4947-8697-dded704324c3 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:38.585869 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6b31a63d-6b87-4947-8697-dded704324c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:38.618672 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 848/1665] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:08:38 2022] POST /identity/v3/projects => generated 385 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:38.622120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6ca89-35c6-4565-8a2d-154507add00a admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:38.626802 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:38.696601 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:38.783708 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6ca89-35c6-4565-8a2d-154507add00a admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:38.792398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b31a63d-6b87-4947-8697-dded704324c3 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:38.819721 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6628c-598f-4793-8896-a95a4556e449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:38.821631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6628c-598f-4793-8896-a95a4556e449 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:38.821915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6628c-598f-4793-8896-a95a4556e449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:38.822192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d6628c-598f-4793-8896-a95a4556e449 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:38.824759 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6628c-598f-4793-8896-a95a4556e449 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:38.827863 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d6628c-598f-4793-8896-a95a4556e449 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:38.880567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 849/1666] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:08:38 2022] POST /identity/v3/users => generated 299 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:08:38.885170 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6628c-598f-4793-8896-a95a4556e449 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:38.920974 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d6628c-598f-4793-8896-a95a4556e449 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:38.936840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9f411-98d0-4ab9-93d0-e4da91f36766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:38.937651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9f411-98d0-4ab9-93d0-e4da91f36766 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:38.938038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9f411-98d0-4ab9-93d0-e4da91f36766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:38.938419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9f411-98d0-4ab9-93d0-e4da91f36766 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:38.939735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9f411-98d0-4ab9-93d0-e4da91f36766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:38.941513 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9f411-98d0-4ab9-93d0-e4da91f36766 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:38.957213 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27317ea0-ebc5-40a0-91b8-f9b516457c92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:38.957867 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27317ea0-ebc5-40a0-91b8-f9b516457c92 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:38.958149 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27317ea0-ebc5-40a0-91b8-f9b516457c92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:38.958420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27317ea0-ebc5-40a0-91b8-f9b516457c92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:38.966353 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 850/1667] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:38.969239 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9f411-98d0-4ab9-93d0-e4da91f36766 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:38.987182 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27317ea0-ebc5-40a0-91b8-f9b516457c92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:38.988659 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27317ea0-ebc5-40a0-91b8-f9b516457c92 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:38.991412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9f411-98d0-4ab9-93d0-e4da91f36766 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:38.998813 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4587c-04d4-4aa3-b54d-5400998b82e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:38.999847 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4587c-04d4-4aa3-b54d-5400998b82e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:39.000294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4587c-04d4-4aa3-b54d-5400998b82e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:39.000819 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a4587c-04d4-4aa3-b54d-5400998b82e2 admin admin] PATH_INFO: `/v3/projects/8418a7ff66e04563bc94f73b64c6af8f/users/a8a633ac7f0046a5b13dea27fe81b30e/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:39.023023 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 818/1668] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:08:39.035777 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4587c-04d4-4aa3-b54d-5400998b82e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8418a7ff66e04563bc94f73b64c6af8f, user_id=a8a633ac7f0046a5b13dea27fe81b30e, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:39.039211 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a4587c-04d4-4aa3-b54d-5400998b82e2 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:39.073996 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 851/1669] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:08:38 2022] PUT /identity/v3/projects/8418a7ff66e04563bc94f73b64c6af8f/users/a8a633ac7f0046a5b13dea27fe81b30e/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:39.085312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27317ea0-ebc5-40a0-91b8-f9b516457c92 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:39.099899 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27317ea0-ebc5-40a0-91b8-f9b516457c92 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:39.216010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ac083-10d2-44c4-8536-e410c48cffb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:39.217898 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ac083-10d2-44c4-8536-e410c48cffb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:39.218182 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ac083-10d2-44c4-8536-e410c48cffb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:39.220385 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ac083-10d2-44c4-8536-e410c48cffb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:39.222504 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ac083-10d2-44c4-8536-e410c48cffb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:39.224383 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459ac083-10d2-44c4-8536-e410c48cffb9 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:39.246597 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 819/1670] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:39.250784 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4587c-04d4-4aa3-b54d-5400998b82e2 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:39.277519 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a4587c-04d4-4aa3-b54d-5400998b82e2 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:39.298542 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3116efd-1831-4279-bb11-2a6ef2749fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:39.299294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3116efd-1831-4279-bb11-2a6ef2749fe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:39.299579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3116efd-1831-4279-bb11-2a6ef2749fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:39.299856 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3116efd-1831-4279-bb11-2a6ef2749fe2 admin admin] PATH_INFO: `/v3/projects/8418a7ff66e04563bc94f73b64c6af8f/users/a8a633ac7f0046a5b13dea27fe81b30e/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:39.307728 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3116efd-1831-4279-bb11-2a6ef2749fe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8418a7ff66e04563bc94f73b64c6af8f, user_id=a8a633ac7f0046a5b13dea27fe81b30e, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:39.320533 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3116efd-1831-4279-bb11-2a6ef2749fe2 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:39.384750 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 852/1671] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:08:39 2022] PUT /identity/v3/projects/8418a7ff66e04563bc94f73b64c6af8f/users/a8a633ac7f0046a5b13dea27fe81b30e/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:39.403262 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f38e70-0964-4e4a-b5c8-d91e3595eabc None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:39.403262 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f38e70-0964-4e4a-b5c8-d91e3595eabc None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:39.403262 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f38e70-0964-4e4a-b5c8-d91e3595eabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:39.540340 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-25f38e70-0964-4e4a-b5c8-d91e3595eabc None None] MFA Rules not processed for user `a8a633ac7f0046a5b13dea27fe81b30e`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:39.541862 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-25f38e70-0964-4e4a-b5c8-d91e3595eabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:39.707507 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:39.789389 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 820/1672] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:39 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:39.791204 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3116efd-1831-4279-bb11-2a6ef2749fe2 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:39.796864 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-d3116efd-1831-4279-bb11-2a6ef2749fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:39.910819 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f38e70-0964-4e4a-b5c8-d91e3595eabc None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:39.912416 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-25f38e70-0964-4e4a-b5c8-d91e3595eabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:40.022916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3116efd-1831-4279-bb11-2a6ef2749fe2 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:40.037690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc9d98-6699-45f6-9e3e-eb92fdc73376 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:40.038810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc9d98-6699-45f6-9e3e-eb92fdc73376 service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:40.039327 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc9d98-6699-45f6-9e3e-eb92fdc73376 service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:40.039788 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc9d98-6699-45f6-9e3e-eb92fdc73376 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:40.096925 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f38e70-0964-4e4a-b5c8-d91e3595eabc None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:40.110178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375be10-cbe5-4b73-b845-214a10b39f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:40.112539 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375be10-cbe5-4b73-b845-214a10b39f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:40.113035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375be10-cbe5-4b73-b845-214a10b39f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:40.113503 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375be10-cbe5-4b73-b845-214a10b39f6c admin admin] PATH_INFO: `/v3/users/5778878319b9470eb17a52d4af060b37` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:40.115762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375be10-cbe5-4b73-b845-214a10b39f6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5778878319b9470eb17a52d4af060b37)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:40.117914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6375be10-cbe5-4b73-b845-214a10b39f6c admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:40.171445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc9d98-6699-45f6-9e3e-eb92fdc73376 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:40.178283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc9d98-6699-45f6-9e3e-eb92fdc73376 service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:40.213984 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-6375be10-cbe5-4b73-b845-214a10b39f6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5778878319b9470eb17a52d4af060b37'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:40.214823 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-6375be10-cbe5-4b73-b845-214a10b39f6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5778878319b9470eb17a52d4af060b37'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:40.217098 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 853/1673] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:08:40.231790 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-6375be10-cbe5-4b73-b845-214a10b39f6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5778878319b9470eb17a52d4af060b37'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:40.246773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-6375be10-cbe5-4b73-b845-214a10b39f6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5778878319b9470eb17a52d4af060b37'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:40.248822 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-6375be10-cbe5-4b73-b845-214a10b39f6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5778878319b9470eb17a52d4af060b37'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:40.277391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 821/1674] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:08:39 2022] DELETE /identity/v3/users/5778878319b9470eb17a52d4af060b37 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:40.282099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c62a8-e2a2-4a50-9d4a-3cf5a0b8d6e0 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:40.282393 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c62a8-e2a2-4a50-9d4a-3cf5a0b8d6e0 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:40.283252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c62a8-e2a2-4a50-9d4a-3cf5a0b8d6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:40.320937 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-cd2c62a8-e2a2-4a50-9d4a-3cf5a0b8d6e0 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:40.322182 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-cd2c62a8-e2a2-4a50-9d4a-3cf5a0b8d6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:40.384736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:40.498992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 854/1675] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:40.554354 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375be10-cbe5-4b73-b845-214a10b39f6c admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:40.556291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6375be10-cbe5-4b73-b845-214a10b39f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:40.697982 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6375be10-cbe5-4b73-b845-214a10b39f6c admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:40.703354 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f4ad7-3886-4f53-91d5-d8aa1975523b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:40.703994 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f4ad7-3886-4f53-91d5-d8aa1975523b service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:40.704321 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f4ad7-3886-4f53-91d5-d8aa1975523b service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:40.704600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f4ad7-3886-4f53-91d5-d8aa1975523b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:40.730915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f4ad7-3886-4f53-91d5-d8aa1975523b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:40.732470 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f4ad7-3886-4f53-91d5-d8aa1975523b service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:40.757981 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 822/1676] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:08:42.531881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c62a8-e2a2-4a50-9d4a-3cf5a0b8d6e0 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:42.534555 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-cd2c62a8-e2a2-4a50-9d4a-3cf5a0b8d6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:42.563121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c62a8-e2a2-4a50-9d4a-3cf5a0b8d6e0 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:42.591961 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0510a2c-09a6-4069-ad40-c7956c2ba88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:42.592787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0510a2c-09a6-4069-ad40-c7956c2ba88f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:42.593147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0510a2c-09a6-4069-ad40-c7956c2ba88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:42.593430 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0510a2c-09a6-4069-ad40-c7956c2ba88f admin admin] PATH_INFO: `/v3/projects/a256f597167b463a95ceadb3eca9c0a3` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:42.595495 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0510a2c-09a6-4069-ad40-c7956c2ba88f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a256f597167b463a95ceadb3eca9c0a3)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:42.597874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0510a2c-09a6-4069-ad40-c7956c2ba88f admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:42.712340 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e0510a2c-09a6-4069-ad40-c7956c2ba88f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a256f597167b463a95ceadb3eca9c0a3', 'request_id': 'req-e0510a2c-09a6-4069-ad40-c7956c2ba88f'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:42.732686 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e0510a2c-09a6-4069-ad40-c7956c2ba88f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a256f597167b463a95ceadb3eca9c0a3', 'request_id': 'req-e0510a2c-09a6-4069-ad40-c7956c2ba88f'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:42.746696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e0510a2c-09a6-4069-ad40-c7956c2ba88f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a256f597167b463a95ceadb3eca9c0a3', 'request_id': 'req-e0510a2c-09a6-4069-ad40-c7956c2ba88f'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:42.778158 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e0510a2c-09a6-4069-ad40-c7956c2ba88f admin admin] The token cache is being invalidate because project a256f597167b463a95ceadb3eca9c0a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:08:42.778493 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e0510a2c-09a6-4069-ad40-c7956c2ba88f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:42.779682 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e0510a2c-09a6-4069-ad40-c7956c2ba88f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:42.787631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 855/1677] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:08:42 2022] DELETE /identity/v3/projects/a256f597167b463a95ceadb3eca9c0a3 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:42.789315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437511f-a60d-4150-95ab-d12d56276e31 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:42.790499 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437511f-a60d-4150-95ab-d12d56276e31 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:42.791170 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437511f-a60d-4150-95ab-d12d56276e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:42.837291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-5437511f-a60d-4150-95ab-d12d56276e31 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:42.838778 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-5437511f-a60d-4150-95ab-d12d56276e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:42.880591 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:43.059446 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 823/1678] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:08:43.068507 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18dd3f-5ea7-4279-9057-bc8a297e30d8 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:43.068898 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18dd3f-5ea7-4279-9057-bc8a297e30d8 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:43.069102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d18dd3f-5ea7-4279-9057-bc8a297e30d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:43.128547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-2d18dd3f-5ea7-4279-9057-bc8a297e30d8 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:43.131358 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-2d18dd3f-5ea7-4279-9057-bc8a297e30d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:43.152734 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 856/1679] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:43.157023 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437511f-a60d-4150-95ab-d12d56276e31 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:43.177686 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437511f-a60d-4150-95ab-d12d56276e31 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:43.218037 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3efda-c4a6-411a-b913-bf2c7c6d0abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:43.219238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3efda-c4a6-411a-b913-bf2c7c6d0abc admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:43.223836 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3efda-c4a6-411a-b913-bf2c7c6d0abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:43.224358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3efda-c4a6-411a-b913-bf2c7c6d0abc admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:43.225276 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3efda-c4a6-411a-b913-bf2c7c6d0abc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:08:43.235151 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3efda-c4a6-411a-b913-bf2c7c6d0abc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:43.242730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3efda-c4a6-411a-b913-bf2c7c6d0abc admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:43.270454 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 824/1680] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:08:43.274838 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18dd3f-5ea7-4279-9057-bc8a297e30d8 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:43.304906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d18dd3f-5ea7-4279-9057-bc8a297e30d8 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:43.323158 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314a371-7a15-4a2e-a085-1bf25c0a5a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:43.323871 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314a371-7a15-4a2e-a085-1bf25c0a5a13 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:43.324145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314a371-7a15-4a2e-a085-1bf25c0a5a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:43.324474 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314a371-7a15-4a2e-a085-1bf25c0a5a13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:43.325177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314a371-7a15-4a2e-a085-1bf25c0a5a13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:43.334393 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314a371-7a15-4a2e-a085-1bf25c0a5a13 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:43.404759 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 857/1681] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:08:43 2022] POST /identity/v3/projects => generated 367 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:43.405534 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3efda-c4a6-411a-b913-bf2c7c6d0abc admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:43.453081 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3efda-c4a6-411a-b913-bf2c7c6d0abc admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:43.677648 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9714e19-11f6-4fe0-8d9b-739d4527db8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:43.678287 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9714e19-11f6-4fe0-8d9b-739d4527db8a admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:43.678560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9714e19-11f6-4fe0-8d9b-739d4527db8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:43.678829 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9714e19-11f6-4fe0-8d9b-739d4527db8a admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:43.679607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9714e19-11f6-4fe0-8d9b-739d4527db8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:43.681554 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9714e19-11f6-4fe0-8d9b-739d4527db8a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:43.713552 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 825/1682] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:08:43 2022] POST /identity/v3/users => generated 284 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:08:43.717207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314a371-7a15-4a2e-a085-1bf25c0a5a13 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:43.735410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314a371-7a15-4a2e-a085-1bf25c0a5a13 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:43.745798 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394db2a0-db03-4ffc-a35f-0ef1a07b69dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:43.746404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394db2a0-db03-4ffc-a35f-0ef1a07b69dc admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:43.746679 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394db2a0-db03-4ffc-a35f-0ef1a07b69dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:43.747132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394db2a0-db03-4ffc-a35f-0ef1a07b69dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:43.747938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394db2a0-db03-4ffc-a35f-0ef1a07b69dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:43.749703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394db2a0-db03-4ffc-a35f-0ef1a07b69dc admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:43.784961 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 858/1683] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:43.789382 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9714e19-11f6-4fe0-8d9b-739d4527db8a admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:43.802869 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9714e19-11f6-4fe0-8d9b-739d4527db8a admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:43.808826 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81750dd2-3fd4-47e4-aaf6-b419b29dbf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:43.812022 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81750dd2-3fd4-47e4-aaf6-b419b29dbf79 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:43.812540 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81750dd2-3fd4-47e4-aaf6-b419b29dbf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:43.813517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81750dd2-3fd4-47e4-aaf6-b419b29dbf79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:43.817412 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81750dd2-3fd4-47e4-aaf6-b419b29dbf79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:43.823433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81750dd2-3fd4-47e4-aaf6-b419b29dbf79 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:43.877676 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 826/1684] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:43.881776 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394db2a0-db03-4ffc-a35f-0ef1a07b69dc admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:43.896096 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394db2a0-db03-4ffc-a35f-0ef1a07b69dc admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:43.904266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42251bf-83c0-4a98-a4cc-159e91eda450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:43.905899 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42251bf-83c0-4a98-a4cc-159e91eda450 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:43.908963 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42251bf-83c0-4a98-a4cc-159e91eda450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:43.909389 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42251bf-83c0-4a98-a4cc-159e91eda450 admin admin] PATH_INFO: `/v3/projects/0b258c97d9dd45b0bf4097edf9ae560c/users/945e4b9be5444bc3864e8d7f906aeb41/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:43.955036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42251bf-83c0-4a98-a4cc-159e91eda450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b258c97d9dd45b0bf4097edf9ae560c, user_id=945e4b9be5444bc3864e8d7f906aeb41, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:43.960902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42251bf-83c0-4a98-a4cc-159e91eda450 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:44.022027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 859/1685] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:08:43 2022] PUT /identity/v3/projects/0b258c97d9dd45b0bf4097edf9ae560c/users/945e4b9be5444bc3864e8d7f906aeb41/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:44.023684 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976ddb7-342a-47d1-afa3-197eb42629f4 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:44.023966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976ddb7-342a-47d1-afa3-197eb42629f4 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:44.024785 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b976ddb7-342a-47d1-afa3-197eb42629f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:44.092003 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-b976ddb7-342a-47d1-afa3-197eb42629f4 None None] MFA Rules not processed for user `945e4b9be5444bc3864e8d7f906aeb41`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:44.093608 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b976ddb7-342a-47d1-afa3-197eb42629f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:44.321528 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42251bf-83c0-4a98-a4cc-159e91eda450 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:44.327573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 827/1686] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:44.450077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:44.458480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa7d86-dfc1-4ffd-ab97-dea1c8393a2a None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:44.458768 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa7d86-dfc1-4ffd-ab97-dea1c8393a2a None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:44.459046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa7d86-dfc1-4ffd-ab97-dea1c8393a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:44.519604 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:44.523778 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42251bf-83c0-4a98-a4cc-159e91eda450 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:44.531743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9d741-14fe-454d-838f-356019ec19e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:44.532592 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9d741-14fe-454d-838f-356019ec19e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:44.533092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9d741-14fe-454d-838f-356019ec19e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:44.534003 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9d741-14fe-454d-838f-356019ec19e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:44.534587 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9d741-14fe-454d-838f-356019ec19e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:44.539030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9d741-14fe-454d-838f-356019ec19e1 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:44.555663 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-69aa7d86-dfc1-4ffd-ab97-dea1c8393a2a None None] MFA Rules not processed for user `bf58157cbc8e49c5bd913c68ff859bfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:44.556963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-69aa7d86-dfc1-4ffd-ab97-dea1c8393a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:44.593087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 860/1687] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:08:44 2022] POST /identity/v3/projects => generated 369 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:44.601425 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9d741-14fe-454d-838f-356019ec19e1 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:44.760981 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9d741-14fe-454d-838f-356019ec19e1 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:44.773957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86eb8e3-7d2e-4f4f-8f3d-01a0b2872a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:44.775519 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86eb8e3-7d2e-4f4f-8f3d-01a0b2872a72 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:44.775998 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86eb8e3-7d2e-4f4f-8f3d-01a0b2872a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:44.776435 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86eb8e3-7d2e-4f4f-8f3d-01a0b2872a72 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:44.777204 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86eb8e3-7d2e-4f4f-8f3d-01a0b2872a72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:44.781152 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86eb8e3-7d2e-4f4f-8f3d-01a0b2872a72 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:44.794921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 828/1688] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:44.796445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ed696-4bd1-4da0-8841-e9f257c75da2 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:44.800109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ed696-4bd1-4da0-8841-e9f257c75da2 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:44.800586 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ed696-4bd1-4da0-8841-e9f257c75da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:44.851987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 861/1689] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:08:44 2022] POST /identity/v3/users => generated 291 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:44.853457 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-1c7ed696-4bd1-4da0-8841-e9f257c75da2 None None] MFA Rules not processed for user `bf58157cbc8e49c5bd913c68ff859bfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:44.853628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86eb8e3-7d2e-4f4f-8f3d-01a0b2872a72 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:44.855457 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1c7ed696-4bd1-4da0-8841-e9f257c75da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:44.905030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86eb8e3-7d2e-4f4f-8f3d-01a0b2872a72 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:44.928446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0beb3a-180c-486d-99e5-7f750aa8034e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:44.931279 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0beb3a-180c-486d-99e5-7f750aa8034e admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:44.931556 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0beb3a-180c-486d-99e5-7f750aa8034e admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:44.931830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0beb3a-180c-486d-99e5-7f750aa8034e admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:44.932493 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0beb3a-180c-486d-99e5-7f750aa8034e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:44.940184 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0beb3a-180c-486d-99e5-7f750aa8034e admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:44.982463 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 862/1690] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:44.987584 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0beb3a-180c-486d-99e5-7f750aa8034e admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:45.032967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0beb3a-180c-486d-99e5-7f750aa8034e admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:45.039729 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 829/1691] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:45.044708 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df594a-97e1-4f9c-8420-5497bd983e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:45.045600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df594a-97e1-4f9c-8420-5497bd983e8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:45.045887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df594a-97e1-4f9c-8420-5497bd983e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:45.046160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df594a-97e1-4f9c-8420-5497bd983e8e admin admin] PATH_INFO: `/v3/projects/99d6b751e9254c23a40173312f865ac1/users/c79592e586c64e1abd574e86cf7afa77/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:45.052983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ed696-4bd1-4da0-8841-e9f257c75da2 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:45.054906 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1c7ed696-4bd1-4da0-8841-e9f257c75da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:45.098064 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df594a-97e1-4f9c-8420-5497bd983e8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=99d6b751e9254c23a40173312f865ac1, user_id=c79592e586c64e1abd574e86cf7afa77, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:45.103520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df594a-97e1-4f9c-8420-5497bd983e8e admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:45.137774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 863/1692] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:08:44 2022] PUT /identity/v3/projects/99d6b751e9254c23a40173312f865ac1/users/c79592e586c64e1abd574e86cf7afa77/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:45.144236 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df594a-97e1-4f9c-8420-5497bd983e8e admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:45.262415 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ed696-4bd1-4da0-8841-e9f257c75da2 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:45.335573 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df594a-97e1-4f9c-8420-5497bd983e8e admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:45.346934 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:45.352522 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e41a6a-4656-4831-8c0f-ab4ff6604921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:45.353248 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e41a6a-4656-4831-8c0f-ab4ff6604921 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:45.353534 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e41a6a-4656-4831-8c0f-ab4ff6604921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:45.353866 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e41a6a-4656-4831-8c0f-ab4ff6604921 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:45.354664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e41a6a-4656-4831-8c0f-ab4ff6604921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:45.357205 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e41a6a-4656-4831-8c0f-ab4ff6604921 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:45.374470 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 864/1693] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:45.375524 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e41a6a-4656-4831-8c0f-ab4ff6604921 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:45.397576 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e41a6a-4656-4831-8c0f-ab4ff6604921 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:45.404263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74232775-fd47-483d-bbad-1add77911e82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:45.404849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74232775-fd47-483d-bbad-1add77911e82 service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:45.405132 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74232775-fd47-483d-bbad-1add77911e82 service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:45.405402 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74232775-fd47-483d-bbad-1add77911e82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:45.405784 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcecac69-b025-46b4-ae1b-b74e853bf7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:45.421036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcecac69-b025-46b4-ae1b-b74e853bf7a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:45.421458 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcecac69-b025-46b4-ae1b-b74e853bf7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:45.421748 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcecac69-b025-46b4-ae1b-b74e853bf7a6 admin admin] PATH_INFO: `/v3/projects/99d6b751e9254c23a40173312f865ac1/users/c79592e586c64e1abd574e86cf7afa77/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:45.436990 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcecac69-b025-46b4-ae1b-b74e853bf7a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99d6b751e9254c23a40173312f865ac1, user_id=c79592e586c64e1abd574e86cf7afa77, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:45.441443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcecac69-b025-46b4-ae1b-b74e853bf7a6 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:45.459372 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 865/1694] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:08:45 2022] PUT /identity/v3/projects/99d6b751e9254c23a40173312f865ac1/users/c79592e586c64e1abd574e86cf7afa77/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:45.472199 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609bdba-fef5-4b69-8b8a-0bb196e49ede None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:45.475538 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609bdba-fef5-4b69-8b8a-0bb196e49ede None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:45.476109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609bdba-fef5-4b69-8b8a-0bb196e49ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:45.492782 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:45.553256 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74232775-fd47-483d-bbad-1add77911e82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:45.553256 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74232775-fd47-483d-bbad-1add77911e82 service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:45.583199 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 830/1695] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:45 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:08:45.584611 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3eab8-9616-423d-9871-6e4949d6938e None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:45.584919 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-6609bdba-fef5-4b69-8b8a-0bb196e49ede None None] MFA Rules not processed for user `c79592e586c64e1abd574e86cf7afa77`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:45.585047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3eab8-9616-423d-9871-6e4949d6938e None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:45.585229 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3eab8-9616-423d-9871-6e4949d6938e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:45.587141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6609bdba-fef5-4b69-8b8a-0bb196e49ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:45.649216 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-6ac3eab8-9616-423d-9871-6e4949d6938e None None] MFA Rules not processed for user `a6e4c04db8af40f288e2b86afb149ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:45.650574 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6ac3eab8-9616-423d-9871-6e4949d6938e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:45.766525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 831/1696] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:45 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:45.775255 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 866/1697] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:45 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:45.779250 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3eab8-9616-423d-9871-6e4949d6938e None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:45.784035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6ac3eab8-9616-423d-9871-6e4949d6938e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:45.787119 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609bdba-fef5-4b69-8b8a-0bb196e49ede None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:45.789195 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6609bdba-fef5-4b69-8b8a-0bb196e49ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:45.914443 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3eab8-9616-423d-9871-6e4949d6938e None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:45.924143 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2584ac2-aa03-4db3-bacc-733972d81ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:45.925079 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2584ac2-aa03-4db3-bacc-733972d81ccf admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:45.925369 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2584ac2-aa03-4db3-bacc-733972d81ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:45.925792 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2584ac2-aa03-4db3-bacc-733972d81ccf admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:45.926450 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2584ac2-aa03-4db3-bacc-733972d81ccf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:45.929408 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2584ac2-aa03-4db3-bacc-733972d81ccf admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:45.930641 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609bdba-fef5-4b69-8b8a-0bb196e49ede None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:45.939690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94953cc3-7869-4244-ac00-ff662fc2357a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:45.940330 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94953cc3-7869-4244-ac00-ff662fc2357a service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:45.940726 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94953cc3-7869-4244-ac00-ff662fc2357a service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:45.941005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94953cc3-7869-4244-ac00-ff662fc2357a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:45.954199 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:45.970580 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94953cc3-7869-4244-ac00-ff662fc2357a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:45.972496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94953cc3-7869-4244-ac00-ff662fc2357a service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:45.975796 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 832/1698] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:08:45 2022] POST /identity/v3/projects => generated 405 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:45.980050 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2584ac2-aa03-4db3-bacc-733972d81ccf admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:45.994447 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 867/1699] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:08:46.193468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2584ac2-aa03-4db3-bacc-733972d81ccf admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:46.222008 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c3601-f76b-4ba3-8e79-c23f24aa33c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:46.222793 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c3601-f76b-4ba3-8e79-c23f24aa33c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:46.223098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c3601-f76b-4ba3-8e79-c23f24aa33c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:46.224400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c3601-f76b-4ba3-8e79-c23f24aa33c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:46.225691 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c3601-f76b-4ba3-8e79-c23f24aa33c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:46.227943 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c3601-f76b-4ba3-8e79-c23f24aa33c2 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:46.280846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94953cc3-7869-4244-ac00-ff662fc2357a service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:46.282114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-94953cc3-7869-4244-ac00-ff662fc2357a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:46.305751 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 833/1700] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:08:45 2022] POST /identity/v3/users => generated 309 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:08:46.306293 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94953cc3-7869-4244-ac00-ff662fc2357a service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:46.309901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c3601-f76b-4ba3-8e79-c23f24aa33c2 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:46.384316 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c3601-f76b-4ba3-8e79-c23f24aa33c2 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:46.403717 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9dafe3-327e-4bd8-bfc7-2ffa3e27836f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:46.404343 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9dafe3-327e-4bd8-bfc7-2ffa3e27836f admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:46.408103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9dafe3-327e-4bd8-bfc7-2ffa3e27836f admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:46.411631 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9dafe3-327e-4bd8-bfc7-2ffa3e27836f admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:46.412469 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9dafe3-327e-4bd8-bfc7-2ffa3e27836f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:46.414109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9dafe3-327e-4bd8-bfc7-2ffa3e27836f admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:46.452813 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 834/1701] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:46.458115 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9dafe3-327e-4bd8-bfc7-2ffa3e27836f admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:46.489836 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9dafe3-327e-4bd8-bfc7-2ffa3e27836f admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:46.491161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd1bb2-be26-49f8-b23a-73574a122a64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:46.492046 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd1bb2-be26-49f8-b23a-73574a122a64 service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:46.500607 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd1bb2-be26-49f8-b23a-73574a122a64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:46.501014 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd1bb2-be26-49f8-b23a-73574a122a64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:46.510722 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-7bfd1bb2-be26-49f8-b23a-73574a122a64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:46.526246 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93585188-db7b-4797-ba00-b9675508bc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:46.531616 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93585188-db7b-4797-ba00-b9675508bc64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:46.531905 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93585188-db7b-4797-ba00-b9675508bc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:46.534247 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93585188-db7b-4797-ba00-b9675508bc64 admin admin] PATH_INFO: `/v3/projects/eaef6c371133417e913918b0bbce9f48/users/1015557945fb4a539a27b1b8fffc8b4c/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:46.641036 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93585188-db7b-4797-ba00-b9675508bc64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaef6c371133417e913918b0bbce9f48, user_id=1015557945fb4a539a27b1b8fffc8b4c, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:46.645855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:46.652445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93585188-db7b-4797-ba00-b9675508bc64 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:46.676804 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 835/1702] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:08:46 2022] PUT /identity/v3/projects/eaef6c371133417e913918b0bbce9f48/users/1015557945fb4a539a27b1b8fffc8b4c/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:46.680756 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93585188-db7b-4797-ba00-b9675508bc64 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:46.698273 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd1bb2-be26-49f8-b23a-73574a122a64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:46.701996 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd1bb2-be26-49f8-b23a-73574a122a64 service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:46.730582 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 868/1703] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:08:46.778627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668efd36-cc10-4fdd-9a60-117ff5eb96b9 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:46.779242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668efd36-cc10-4fdd-9a60-117ff5eb96b9 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:46.779537 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668efd36-cc10-4fdd-9a60-117ff5eb96b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:46.831177 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93585188-db7b-4797-ba00-b9675508bc64 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:46.844777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640084e-7a22-456f-aa20-3452d29522c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:46.845619 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640084e-7a22-456f-aa20-3452d29522c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:46.845912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640084e-7a22-456f-aa20-3452d29522c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:46.846185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640084e-7a22-456f-aa20-3452d29522c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:46.846856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640084e-7a22-456f-aa20-3452d29522c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:46.848491 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640084e-7a22-456f-aa20-3452d29522c6 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:46.875772 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 836/1704] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:08:46.875965 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-668efd36-cc10-4fdd-9a60-117ff5eb96b9 None None] MFA Rules not processed for user `10fcb673c54e484a88a0dabe7b8edb15`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:46.875965 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-668efd36-cc10-4fdd-9a60-117ff5eb96b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:46.877106 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640084e-7a22-456f-aa20-3452d29522c6 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:46.900849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640084e-7a22-456f-aa20-3452d29522c6 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:46.924972 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d61d9-057d-4f73-abd6-60f176e3532a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:46.925557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d61d9-057d-4f73-abd6-60f176e3532a admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:46.925986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d61d9-057d-4f73-abd6-60f176e3532a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:46.926458 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2d61d9-057d-4f73-abd6-60f176e3532a admin admin] PATH_INFO: `/v3/projects/eaef6c371133417e913918b0bbce9f48/users/1015557945fb4a539a27b1b8fffc8b4c/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:46.935118 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d61d9-057d-4f73-abd6-60f176e3532a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaef6c371133417e913918b0bbce9f48, user_id=1015557945fb4a539a27b1b8fffc8b4c, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:46.938088 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2d61d9-057d-4f73-abd6-60f176e3532a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:46.964346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 837/1705] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:08:46 2022] PUT /identity/v3/projects/eaef6c371133417e913918b0bbce9f48/users/1015557945fb4a539a27b1b8fffc8b4c/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:47.070400 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 869/1706] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:47.099403 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d61d9-057d-4f73-abd6-60f176e3532a admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:47.101059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-bf2d61d9-057d-4f73-abd6-60f176e3532a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:47.211353 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70258e-0a5c-4308-8a50-d856ef4f845d None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:47.211914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70258e-0a5c-4308-8a50-d856ef4f845d None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:47.212289 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b70258e-0a5c-4308-8a50-d856ef4f845d None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:47.213932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 870/1707] 10.222.0.36 () {54 vars in 912 bytes} [Fri Sep 2 09:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:08:47.221618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c153cc-b1a6-42a9-9b80-abc5fa6360fa None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:47.222590 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c153cc-b1a6-42a9-9b80-abc5fa6360fa None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:47.222590 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c153cc-b1a6-42a9-9b80-abc5fa6360fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:47.237678 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:47.371507 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-b6c153cc-b1a6-42a9-9b80-abc5fa6360fa None None] MFA Rules not processed for user `07a825fddbc04f918bcd631976038cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:47.373578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-b6c153cc-b1a6-42a9-9b80-abc5fa6360fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:47.427331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2d61d9-057d-4f73-abd6-60f176e3532a admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:47.448730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35422e87-af93-40c9-bd45-b64c6fda8147 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:47.449359 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35422e87-af93-40c9-bd45-b64c6fda8147 service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:47.452617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35422e87-af93-40c9-bd45-b64c6fda8147 service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:47.452879 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35422e87-af93-40c9-bd45-b64c6fda8147 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:47.571145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 871/1708] 10.222.0.36 () {58 vars in 1009 bytes} [Fri Sep 2 09:08:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:08:47.574706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67623432-cd60-41aa-b6cc-82a8fe86786f None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:47.578512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67623432-cd60-41aa-b6cc-82a8fe86786f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:47.578512 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67623432-cd60-41aa-b6cc-82a8fe86786f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:47.637565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35422e87-af93-40c9-bd45-b64c6fda8147 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:47.640208 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35422e87-af93-40c9-bd45-b64c6fda8147 service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:47.668297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-67623432-cd60-41aa-b6cc-82a8fe86786f None None] MFA Rules not processed for user `3adaaf8d979d4cf28456812f9dcfa809`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:47.669424 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 838/1709] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:08:47.675594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-67623432-cd60-41aa-b6cc-82a8fe86786f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:47.692447 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35422e87-af93-40c9-bd45-b64c6fda8147 service nova] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:47.695522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-35422e87-af93-40c9-bd45-b64c6fda8147 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:47.742136 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35422e87-af93-40c9-bd45-b64c6fda8147 service nova] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:47.755894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2bb2d-d851-4a65-823c-1d6471f2a1cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:47.756598 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2bb2d-d851-4a65-823c-1d6471f2a1cd service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:47.756886 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2bb2d-d851-4a65-823c-1d6471f2a1cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:47.757161 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2bb2d-d851-4a65-823c-1d6471f2a1cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:47.790476 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2bb2d-d851-4a65-823c-1d6471f2a1cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:47.792596 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2bb2d-d851-4a65-823c-1d6471f2a1cd service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:47.824629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 839/1710] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:08:47.892457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:47.907112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 872/1711] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:47.921878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03e9be-b170-4c11-8c75-d0bdd2ca79d6 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:47.922460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03e9be-b170-4c11-8c75-d0bdd2ca79d6 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:47.922882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a03e9be-b170-4c11-8c75-d0bdd2ca79d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:47.953366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-7a03e9be-b170-4c11-8c75-d0bdd2ca79d6 None None] MFA Rules not processed for user `3adaaf8d979d4cf28456812f9dcfa809`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:47.956402 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7a03e9be-b170-4c11-8c75-d0bdd2ca79d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:47.972460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 840/1712] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:48.011655 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67623432-cd60-41aa-b6cc-82a8fe86786f None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:48.037791 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67623432-cd60-41aa-b6cc-82a8fe86786f None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:48.058671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9343d-d81d-4522-8315-42861b4998e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:48.059506 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9343d-d81d-4522-8315-42861b4998e9 service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:48.059851 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9343d-d81d-4522-8315-42861b4998e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:48.060186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9343d-d81d-4522-8315-42861b4998e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:48.081383 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9343d-d81d-4522-8315-42861b4998e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:48.085833 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9343d-d81d-4522-8315-42861b4998e9 service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:48.109776 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 873/1713] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:08:48.592959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add4d61-3efa-4ec0-b0d6-a4a7116e7a7b None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:48.593241 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add4d61-3efa-4ec0-b0d6-a4a7116e7a7b None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:48.597031 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add4d61-3efa-4ec0-b0d6-a4a7116e7a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:48.665945 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-4add4d61-3efa-4ec0-b0d6-a4a7116e7a7b None None] MFA Rules not processed for user `4210f948dc7e4a5bac61d10eb6a62eee`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:48.667269 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-4add4d61-3efa-4ec0-b0d6-a4a7116e7a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:48.776651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94bda2-99df-45f6-809e-51dbc5933c5b None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:48.777285 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94bda2-99df-45f6-809e-51dbc5933c5b None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:48.777745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94bda2-99df-45f6-809e-51dbc5933c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:48.834386 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-fb94bda2-99df-45f6-809e-51dbc5933c5b None None] MFA Rules not processed for user `1015557945fb4a539a27b1b8fffc8b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:48.835794 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-fb94bda2-99df-45f6-809e-51dbc5933c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:48.850813 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 841/1714] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:49.009030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 874/1715] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:48 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:49.346602 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f046b-0ca3-4f36-9df3-d7414a04266f None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:49.347309 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f046b-0ca3-4f36-9df3-d7414a04266f None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:49.351429 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5f046b-0ca3-4f36-9df3-d7414a04266f None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:49.351429 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 842/1716] 10.222.0.36 () {54 vars in 912 bytes} [Fri Sep 2 09:08:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:08:49.360827 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16498925-bdb4-45a1-a3cc-aa01ce4a2f1f None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:49.361174 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16498925-bdb4-45a1-a3cc-aa01ce4a2f1f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:49.361516 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16498925-bdb4-45a1-a3cc-aa01ce4a2f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:49.403904 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-16498925-bdb4-45a1-a3cc-aa01ce4a2f1f None None] MFA Rules not processed for user `07a825fddbc04f918bcd631976038cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:49.405619 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-16498925-bdb4-45a1-a3cc-aa01ce4a2f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:49.425298 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 875/1717] 10.222.0.36 () {58 vars in 1009 bytes} [Fri Sep 2 09:08:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:49.432151 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f046b-0ca3-4f36-9df3-d7414a04266f None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:49.502482 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16498925-bdb4-45a1-a3cc-aa01ce4a2f1f None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:49.514176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16498925-bdb4-45a1-a3cc-aa01ce4a2f1f None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:49.533055 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa24df-2972-4bfd-9b24-26efd4f6a393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:49.535517 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa24df-2972-4bfd-9b24-26efd4f6a393 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:49.535886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa24df-2972-4bfd-9b24-26efd4f6a393 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:49.536258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa24df-2972-4bfd-9b24-26efd4f6a393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:49.565557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:49.568723 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa24df-2972-4bfd-9b24-26efd4f6a393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:49.570300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa24df-2972-4bfd-9b24-26efd4f6a393 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:49.597110 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 876/1718] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:08:49.623683 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5f046b-0ca3-4f36-9df3-d7414a04266f None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:49.635374 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027ff42-314f-4be2-b4ce-64c5392d924d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:49.636141 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027ff42-314f-4be2-b4ce-64c5392d924d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:49.636541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027ff42-314f-4be2-b4ce-64c5392d924d admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:49.636914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027ff42-314f-4be2-b4ce-64c5392d924d admin admin] PATH_INFO: `/v3/users/945e4b9be5444bc3864e8d7f906aeb41` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:49.638996 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027ff42-314f-4be2-b4ce-64c5392d924d admin admin] RBAC: Authorizing `identity:delete_user(user_id=945e4b9be5444bc3864e8d7f906aeb41)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:49.641090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027ff42-314f-4be2-b4ce-64c5392d924d admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:49.740777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4027ff42-314f-4be2-b4ce-64c5392d924d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '945e4b9be5444bc3864e8d7f906aeb41'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:49.741758 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4027ff42-314f-4be2-b4ce-64c5392d924d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '945e4b9be5444bc3864e8d7f906aeb41'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:49.772056 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4027ff42-314f-4be2-b4ce-64c5392d924d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '945e4b9be5444bc3864e8d7f906aeb41'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:49.791745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4027ff42-314f-4be2-b4ce-64c5392d924d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '945e4b9be5444bc3864e8d7f906aeb41'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:49.799635 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4027ff42-314f-4be2-b4ce-64c5392d924d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '945e4b9be5444bc3864e8d7f906aeb41'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:49.830713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f9a68-955c-4267-83b7-3fd76b8cd56a None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:49.830999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f9a68-955c-4267-83b7-3fd76b8cd56a None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:49.831312 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78f9a68-955c-4267-83b7-3fd76b8cd56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:49.837710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 843/1719] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:08:49 2022] DELETE /identity/v3/users/945e4b9be5444bc3864e8d7f906aeb41 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:49.894162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-c78f9a68-955c-4267-83b7-3fd76b8cd56a None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:49.897558 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c78f9a68-955c-4267-83b7-3fd76b8cd56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:49.950494 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027ff42-314f-4be2-b4ce-64c5392d924d admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:49.952287 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-4027ff42-314f-4be2-b4ce-64c5392d924d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:50.051479 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 877/1720] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:50.070421 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f9a68-955c-4267-83b7-3fd76b8cd56a None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:50.073620 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c78f9a68-955c-4267-83b7-3fd76b8cd56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:50.198744 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027ff42-314f-4be2-b4ce-64c5392d924d admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:50.232403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:50.234458 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c186a-eeee-4182-9f22-69a5aa633612 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:50.235188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c186a-eeee-4182-9f22-69a5aa633612 service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:50.235590 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c186a-eeee-4182-9f22-69a5aa633612 service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:50.235936 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c186a-eeee-4182-9f22-69a5aa633612 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:50.241358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-3d8c186a-eeee-4182-9f22-69a5aa633612 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:50.294219 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78f9a68-955c-4267-83b7-3fd76b8cd56a None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:50.317853 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2b9eb-d8fa-4c83-ba4c-3a062166363d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a0f0fdbbd20492d8c35ec4b73404632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:50.318550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2b9eb-d8fa-4c83-ba4c-3a062166363d service placement] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:50.320359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2b9eb-d8fa-4c83-ba4c-3a062166363d service placement] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:50.320715 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2b9eb-d8fa-4c83-ba4c-3a062166363d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:50.326085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-02e2b9eb-d8fa-4c83-ba4c-3a062166363d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:50.365604 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2b9eb-d8fa-4c83-ba4c-3a062166363d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:50.369069 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2b9eb-d8fa-4c83-ba4c-3a062166363d service placement] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:50.406376 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 878/1721] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:08:50.416942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2b9eb-d8fa-4c83-ba4c-3a062166363d service placement] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:50.419087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-02e2b9eb-d8fa-4c83-ba4c-3a062166363d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:50.438259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:50.449794 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c186a-eeee-4182-9f22-69a5aa633612 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:50.451456 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c186a-eeee-4182-9f22-69a5aa633612 service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:50.480956 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 844/1722] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:08:50.577058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2b9eb-d8fa-4c83-ba4c-3a062166363d service placement] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:50.585770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f03842-aad8-4209-99b1-ed6bd149c7f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:50.586483 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f03842-aad8-4209-99b1-ed6bd149c7f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:50.586763 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f03842-aad8-4209-99b1-ed6bd149c7f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:50.587032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f03842-aad8-4209-99b1-ed6bd149c7f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:50.608832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f03842-aad8-4209-99b1-ed6bd149c7f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:50.610293 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f03842-aad8-4209-99b1-ed6bd149c7f0 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:50.638707 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 879/1723] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:08:50.920463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0737b-fdd5-4f2b-bf78-27b9d3bf5fb5 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:50.920938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0737b-fdd5-4f2b-bf78-27b9d3bf5fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:50.921369 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0737b-fdd5-4f2b-bf78-27b9d3bf5fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:50.942980 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a86b69-b76a-4bd6-a553-ff560c207e51 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:50.943459 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a86b69-b76a-4bd6-a553-ff560c207e51 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:50.943914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a86b69-b76a-4bd6-a553-ff560c207e51 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:50.945851 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 880/1724] 10.222.0.36 () {54 vars in 921 bytes} [Fri Sep 2 09:08:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:08:50.964161 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-94b0737b-fdd5-4f2b-bf78-27b9d3bf5fb5 None None] MFA Rules not processed for user `bf58157cbc8e49c5bd913c68ff859bfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:50.965695 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-94b0737b-fdd5-4f2b-bf78-27b9d3bf5fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:51.034352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f8033-b179-40cf-a487-ddf35232f2ab None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:51.034908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f8033-b179-40cf-a487-ddf35232f2ab None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:51.035430 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f8033-b179-40cf-a487-ddf35232f2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:51.111278 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-869f8033-b179-40cf-a487-ddf35232f2ab None None] MFA Rules not processed for user `6a0f0fdbbd20492d8c35ec4b73404632`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:51.113304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-869f8033-b179-40cf-a487-ddf35232f2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:51.153010 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 881/1725] 10.222.0.36 () {58 vars in 1018 bytes} [Fri Sep 2 09:08:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:08:51.205457 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 845/1726] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:08:51.214827 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12718e-ac8c-4b91-9756-3251fcb21239 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:51.215148 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12718e-ac8c-4b91-9756-3251fcb21239 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:51.215425 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12718e-ac8c-4b91-9756-3251fcb21239 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:51.217116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 882/1727] 10.222.0.36 () {54 vars in 961 bytes} [Fri Sep 2 09:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:08:51.223469 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278e362-c278-41f3-80f8-0e71b8f84adf None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:51.223984 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278e362-c278-41f3-80f8-0e71b8f84adf None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:51.224403 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278e362-c278-41f3-80f8-0e71b8f84adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:51.292676 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-2278e362-c278-41f3-80f8-0e71b8f84adf None None] MFA Rules not processed for user `6a0f0fdbbd20492d8c35ec4b73404632`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:51.293931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-2278e362-c278-41f3-80f8-0e71b8f84adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:51.330667 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 846/1728] 10.222.0.36 () {58 vars in 1058 bytes} [Fri Sep 2 09:08:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:51.339369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12718e-ac8c-4b91-9756-3251fcb21239 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:51.369237 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12718e-ac8c-4b91-9756-3251fcb21239 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:51.382369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c0810-d453-430b-8c60-f91f1bac1151 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a0f0fdbbd20492d8c35ec4b73404632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:51.382991 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c0810-d453-430b-8c60-f91f1bac1151 service placement] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:51.383639 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c0810-d453-430b-8c60-f91f1bac1151 service placement] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:51.384315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6c0810-d453-430b-8c60-f91f1bac1151 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:51.405059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c0810-d453-430b-8c60-f91f1bac1151 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:51.406775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6c0810-d453-430b-8c60-f91f1bac1151 service placement] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:51.447556 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 883/1729] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:08:52.200201 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c91d58-5b3b-4dd2-ac54-852641d0bbbc None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:52.200201 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c91d58-5b3b-4dd2-ac54-852641d0bbbc None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:52.200201 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c91d58-5b3b-4dd2-ac54-852641d0bbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:52.276013 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-18c91d58-5b3b-4dd2-ac54-852641d0bbbc None None] MFA Rules not processed for user `e8b5ce2249a84c01ab57c03b656b2949`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:52.277441 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-18c91d58-5b3b-4dd2-ac54-852641d0bbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:52.480329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 847/1730] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:52.536296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c0810-d453-430b-8c60-f91f1bac1151 service placement] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:52.537790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6f6c0810-d453-430b-8c60-f91f1bac1151 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:52.579264 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6c0810-d453-430b-8c60-f91f1bac1151 service placement] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:52.603415 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6557b-670c-432a-9e44-e568d2eb032a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:52.604035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6557b-670c-432a-9e44-e568d2eb032a service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:52.604816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6557b-670c-432a-9e44-e568d2eb032a service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:52.606046 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6557b-670c-432a-9e44-e568d2eb032a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:52.635100 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6557b-670c-432a-9e44-e568d2eb032a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:52.637051 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe6557b-670c-432a-9e44-e568d2eb032a service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:52.681765 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 884/1731] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:08:52.970750 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c91d58-5b3b-4dd2-ac54-852641d0bbbc None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:52.972476 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-18c91d58-5b3b-4dd2-ac54-852641d0bbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:52.998764 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c91d58-5b3b-4dd2-ac54-852641d0bbbc None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:53.009282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b5fbb-3bb3-419d-ae3e-0e0c72433726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:53.010234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b5fbb-3bb3-419d-ae3e-0e0c72433726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:53.010637 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b5fbb-3bb3-419d-ae3e-0e0c72433726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:53.012471 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b5fbb-3bb3-419d-ae3e-0e0c72433726 admin admin] PATH_INFO: `/v3/users/f4ffc9fc81a74eeba1ea05488fda1f41` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:53.015915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b5fbb-3bb3-419d-ae3e-0e0c72433726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4ffc9fc81a74eeba1ea05488fda1f41)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:53.018543 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b5fbb-3bb3-419d-ae3e-0e0c72433726 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:53.141582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-660b5fbb-3bb3-419d-ae3e-0e0c72433726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4ffc9fc81a74eeba1ea05488fda1f41'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:53.143037 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-660b5fbb-3bb3-419d-ae3e-0e0c72433726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4ffc9fc81a74eeba1ea05488fda1f41'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:53.183297 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-660b5fbb-3bb3-419d-ae3e-0e0c72433726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4ffc9fc81a74eeba1ea05488fda1f41'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:53.197661 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6557b-670c-432a-9e44-e568d2eb032a service nova] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:53.199341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-ebe6557b-670c-432a-9e44-e568d2eb032a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:53.208354 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-660b5fbb-3bb3-419d-ae3e-0e0c72433726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4ffc9fc81a74eeba1ea05488fda1f41'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:53.220848 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-660b5fbb-3bb3-419d-ae3e-0e0c72433726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4ffc9fc81a74eeba1ea05488fda1f41'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:53.228184 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe6557b-670c-432a-9e44-e568d2eb032a service nova] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:53.234192 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:53.243801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf23c0-467e-40b7-82ee-7878e0890826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:53.245033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf23c0-467e-40b7-82ee-7878e0890826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:53.245529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf23c0-467e-40b7-82ee-7878e0890826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:53.246034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf23c0-467e-40b7-82ee-7878e0890826 admin admin] PATH_INFO: `/v3/projects/0b258c97d9dd45b0bf4097edf9ae560c` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:53.247734 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf23c0-467e-40b7-82ee-7878e0890826 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b258c97d9dd45b0bf4097edf9ae560c)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:53.260193 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf23c0-467e-40b7-82ee-7878e0890826 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:53.265232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 848/1732] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:08:52 2022] DELETE /identity/v3/users/f4ffc9fc81a74eeba1ea05488fda1f41 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:53.275673 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07236881-a642-4e63-9780-24c9ba1890b6 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:53.276160 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07236881-a642-4e63-9780-24c9ba1890b6 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:53.276593 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07236881-a642-4e63-9780-24c9ba1890b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:53.305963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-07236881-a642-4e63-9780-24c9ba1890b6 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:53.308251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-07236881-a642-4e63-9780-24c9ba1890b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:53.389116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9ddf23c0-467e-40b7-82ee-7878e0890826 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b258c97d9dd45b0bf4097edf9ae560c', 'request_id': 'req-9ddf23c0-467e-40b7-82ee-7878e0890826'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:53.399367 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9ddf23c0-467e-40b7-82ee-7878e0890826 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b258c97d9dd45b0bf4097edf9ae560c', 'request_id': 'req-9ddf23c0-467e-40b7-82ee-7878e0890826'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:53.400137 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 849/1733] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:53.413774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9ddf23c0-467e-40b7-82ee-7878e0890826 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b258c97d9dd45b0bf4097edf9ae560c', 'request_id': 'req-9ddf23c0-467e-40b7-82ee-7878e0890826'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:53.444833 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9ddf23c0-467e-40b7-82ee-7878e0890826 admin admin] The token cache is being invalidate because project 0b258c97d9dd45b0bf4097edf9ae560c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:08:53.445122 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9ddf23c0-467e-40b7-82ee-7878e0890826 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:53.447436 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9ddf23c0-467e-40b7-82ee-7878e0890826 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:53.453556 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 885/1734] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:08:53 2022] DELETE /identity/v3/projects/0b258c97d9dd45b0bf4097edf9ae560c => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:53.455342 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07236881-a642-4e63-9780-24c9ba1890b6 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:53.457022 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-07236881-a642-4e63-9780-24c9ba1890b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:53.484831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07236881-a642-4e63-9780-24c9ba1890b6 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:53.505361 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf23c0-467e-40b7-82ee-7878e0890826 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:53.507278 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-9ddf23c0-467e-40b7-82ee-7878e0890826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:53.638280 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:53.693866 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf23c0-467e-40b7-82ee-7878e0890826 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:53.698273 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d8ccc-a380-4104-b266-8548b732d226 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:53.699155 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d8ccc-a380-4104-b266-8548b732d226 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:53.699603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d8ccc-a380-4104-b266-8548b732d226 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:53.700030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d8ccc-a380-4104-b266-8548b732d226 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:53.704176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-579d8ccc-a380-4104-b266-8548b732d226 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:53.731373 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e79bd-9251-4418-b07e-ddc7ab88ccb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:53.732034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e79bd-9251-4418-b07e-ddc7ab88ccb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:53.732320 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e79bd-9251-4418-b07e-ddc7ab88ccb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:53.732610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39e79bd-9251-4418-b07e-ddc7ab88ccb4 admin admin] PATH_INFO: `/v3/users/c79592e586c64e1abd574e86cf7afa77` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:53.734803 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39e79bd-9251-4418-b07e-ddc7ab88ccb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c79592e586c64e1abd574e86cf7afa77)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:53.740456 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39e79bd-9251-4418-b07e-ddc7ab88ccb4 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:53.864197 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b39e79bd-9251-4418-b07e-ddc7ab88ccb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c79592e586c64e1abd574e86cf7afa77'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:53.866852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b39e79bd-9251-4418-b07e-ddc7ab88ccb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c79592e586c64e1abd574e86cf7afa77'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:53.899198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d8ccc-a380-4104-b266-8548b732d226 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:53.901116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d8ccc-a380-4104-b266-8548b732d226 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:53.903441 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b39e79bd-9251-4418-b07e-ddc7ab88ccb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c79592e586c64e1abd574e86cf7afa77'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:53.919742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b39e79bd-9251-4418-b07e-ddc7ab88ccb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c79592e586c64e1abd574e86cf7afa77'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:53.920338 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 886/1735] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:08:53.920546 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b39e79bd-9251-4418-b07e-ddc7ab88ccb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c79592e586c64e1abd574e86cf7afa77'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:53.952166 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 850/1736] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:08:53 2022] DELETE /identity/v3/users/c79592e586c64e1abd574e86cf7afa77 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:08:54.544188 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc4b27-543f-4475-8fad-95d3d4a3e1b8 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:54.544472 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc4b27-543f-4475-8fad-95d3d4a3e1b8 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:54.544915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc4b27-543f-4475-8fad-95d3d4a3e1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:54.633561 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-b2dc4b27-543f-4475-8fad-95d3d4a3e1b8 None None] MFA Rules not processed for user `76ebecce283f491e82a961d7f739878f`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:54.635682 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-b2dc4b27-543f-4475-8fad-95d3d4a3e1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:54.765545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 887/1737] 10.222.0.36 () {58 vars in 1009 bytes} [Fri Sep 2 09:08:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:55.060032 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e79bd-9251-4418-b07e-ddc7ab88ccb4 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:55.061790 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b39e79bd-9251-4418-b07e-ddc7ab88ccb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:55.070840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc4b27-543f-4475-8fad-95d3d4a3e1b8 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:55.072751 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-b2dc4b27-543f-4475-8fad-95d3d4a3e1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:55.154930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:55.320806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:55.335588 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc4b27-543f-4475-8fad-95d3d4a3e1b8 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:55.364501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36284678-daf3-46b9-b68b-8179ed852302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:55.365164 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36284678-daf3-46b9-b68b-8179ed852302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:55.365784 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36284678-daf3-46b9-b68b-8179ed852302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:55.366083 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36284678-daf3-46b9-b68b-8179ed852302 admin admin] PATH_INFO: `/v3/projects/0bf099dbeecf43569b5e7ff0cdd6ac73` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:55.372635 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36284678-daf3-46b9-b68b-8179ed852302 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bf099dbeecf43569b5e7ff0cdd6ac73)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:55.379504 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36284678-daf3-46b9-b68b-8179ed852302 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:55.415634 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39e79bd-9251-4418-b07e-ddc7ab88ccb4 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:55.432679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6bce5-c7f2-463b-8821-6a8d0ec30492 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92318948dcf1407b9fc8aa5c199787ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:55.433478 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6bce5-c7f2-463b-8821-6a8d0ec30492 service swift] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:55.433762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6bce5-c7f2-463b-8821-6a8d0ec30492 service swift] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:55.434058 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6bce5-c7f2-463b-8821-6a8d0ec30492 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:55.459317 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6bce5-c7f2-463b-8821-6a8d0ec30492 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:55.463301 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6bce5-c7f2-463b-8821-6a8d0ec30492 service swift] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:55.507004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 851/1738] 10.222.0.36 () {60 vars in 1384 bytes} [Fri Sep 2 09:08:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:08:55.509925 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-36284678-daf3-46b9-b68b-8179ed852302 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bf099dbeecf43569b5e7ff0cdd6ac73', 'request_id': 'req-36284678-daf3-46b9-b68b-8179ed852302'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:55.522299 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-36284678-daf3-46b9-b68b-8179ed852302 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bf099dbeecf43569b5e7ff0cdd6ac73', 'request_id': 'req-36284678-daf3-46b9-b68b-8179ed852302'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:55.534621 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-36284678-daf3-46b9-b68b-8179ed852302 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bf099dbeecf43569b5e7ff0cdd6ac73', 'request_id': 'req-36284678-daf3-46b9-b68b-8179ed852302'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:55.563399 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-36284678-daf3-46b9-b68b-8179ed852302 admin admin] The token cache is being invalidate because project 0bf099dbeecf43569b5e7ff0cdd6ac73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:08:55.566050 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-36284678-daf3-46b9-b68b-8179ed852302 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:55.567106 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-36284678-daf3-46b9-b68b-8179ed852302 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:55.572141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 888/1739] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:08:55 2022] DELETE /identity/v3/projects/0bf099dbeecf43569b5e7ff0cdd6ac73 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:55.575643 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6bce5-c7f2-463b-8821-6a8d0ec30492 service swift] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:55.593161 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6bce5-c7f2-463b-8821-6a8d0ec30492 service swift] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:55.780073 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6364b-524b-4e10-9cd4-473e5dd2970b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:55.780745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6364b-524b-4e10-9cd4-473e5dd2970b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:55.781040 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6364b-524b-4e10-9cd4-473e5dd2970b admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:55.781383 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6364b-524b-4e10-9cd4-473e5dd2970b admin admin] PATH_INFO: `/v3/users/a8a633ac7f0046a5b13dea27fe81b30e` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:55.790500 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6364b-524b-4e10-9cd4-473e5dd2970b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8a633ac7f0046a5b13dea27fe81b30e)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:55.792505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6364b-524b-4e10-9cd4-473e5dd2970b admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:55.958969 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9ea6364b-524b-4e10-9cd4-473e5dd2970b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8a633ac7f0046a5b13dea27fe81b30e'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:55.960070 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9ea6364b-524b-4e10-9cd4-473e5dd2970b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8a633ac7f0046a5b13dea27fe81b30e'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:55.977815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9ea6364b-524b-4e10-9cd4-473e5dd2970b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8a633ac7f0046a5b13dea27fe81b30e'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:56.007858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9ea6364b-524b-4e10-9cd4-473e5dd2970b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8a633ac7f0046a5b13dea27fe81b30e'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:56.009048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9ea6364b-524b-4e10-9cd4-473e5dd2970b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8a633ac7f0046a5b13dea27fe81b30e'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:56.053134 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 852/1740] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:08:55 2022] DELETE /identity/v3/users/a8a633ac7f0046a5b13dea27fe81b30e => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:56.073241 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9130e9-c4d3-4a72-a190-f31f607115fe None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:56.075536 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9130e9-c4d3-4a72-a190-f31f607115fe None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:56.077781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9130e9-c4d3-4a72-a190-f31f607115fe None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:56.080356 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 889/1741] 10.222.0.36 () {54 vars in 921 bytes} [Fri Sep 2 09:08:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:08:56.322496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc686e95-aa60-484b-825c-e556634d97ee None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:56.323311 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc686e95-aa60-484b-825c-e556634d97ee None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:56.323311 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc686e95-aa60-484b-825c-e556634d97ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:56.342078 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:56.473609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-bc686e95-aa60-484b-825c-e556634d97ee None None] MFA Rules not processed for user `6a0f0fdbbd20492d8c35ec4b73404632`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:56.474900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-bc686e95-aa60-484b-825c-e556634d97ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:56.692747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 853/1742] 10.222.0.36 () {58 vars in 1018 bytes} [Fri Sep 2 09:08:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:56.780820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9130e9-c4d3-4a72-a190-f31f607115fe None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:56.783904 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8a9130e9-c4d3-4a72-a190-f31f607115fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:56.817839 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9130e9-c4d3-4a72-a190-f31f607115fe None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:56.834257 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f6ec9-c849-4d53-83ec-3835721d9181 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a0f0fdbbd20492d8c35ec4b73404632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:56.835111 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f6ec9-c849-4d53-83ec-3835721d9181 service placement] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:56.835417 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f6ec9-c849-4d53-83ec-3835721d9181 service placement] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:56.835758 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f6ec9-c849-4d53-83ec-3835721d9181 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:56.872883 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f6ec9-c849-4d53-83ec-3835721d9181 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:56.875892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f6ec9-c849-4d53-83ec-3835721d9181 service placement] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:56.917874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 890/1743] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:08:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:08:57.011260 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc686e95-aa60-484b-825c-e556634d97ee None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:57.013357 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-bc686e95-aa60-484b-825c-e556634d97ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:57.279014 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc686e95-aa60-484b-825c-e556634d97ee None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:57.293676 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3004725-ecdc-4617-9b4a-41d97c4076cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:57.294614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3004725-ecdc-4617-9b4a-41d97c4076cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:57.294956 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3004725-ecdc-4617-9b4a-41d97c4076cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:57.297372 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3004725-ecdc-4617-9b4a-41d97c4076cf admin admin] PATH_INFO: `/v3/users/a6e4c04db8af40f288e2b86afb149ec4` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:57.305325 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3004725-ecdc-4617-9b4a-41d97c4076cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6e4c04db8af40f288e2b86afb149ec4)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:57.308333 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3004725-ecdc-4617-9b4a-41d97c4076cf admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:57.475497 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d3004725-ecdc-4617-9b4a-41d97c4076cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6e4c04db8af40f288e2b86afb149ec4'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:57.477586 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d3004725-ecdc-4617-9b4a-41d97c4076cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6e4c04db8af40f288e2b86afb149ec4'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:57.515323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d3004725-ecdc-4617-9b4a-41d97c4076cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6e4c04db8af40f288e2b86afb149ec4'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:57.536052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d3004725-ecdc-4617-9b4a-41d97c4076cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6e4c04db8af40f288e2b86afb149ec4'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:57.536908 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d3004725-ecdc-4617-9b4a-41d97c4076cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6e4c04db8af40f288e2b86afb149ec4'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:57.567598 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 854/1744] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:08:57 2022] DELETE /identity/v3/users/a6e4c04db8af40f288e2b86afb149ec4 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:57.574665 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f6ec9-c849-4d53-83ec-3835721d9181 service placement] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:57.576620 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-cd5f6ec9-c849-4d53-83ec-3835721d9181 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:57.602429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f6ec9-c849-4d53-83ec-3835721d9181 service placement] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:57.747240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb7dfc-7695-499b-9795-807b32b92a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:57.755730 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb7dfc-7695-499b-9795-807b32b92a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:57.756165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb7dfc-7695-499b-9795-807b32b92a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:57.756450 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb7dfc-7695-499b-9795-807b32b92a39 admin admin] PATH_INFO: `/v3/projects/99d6b751e9254c23a40173312f865ac1` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:57.759866 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb7dfc-7695-499b-9795-807b32b92a39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99d6b751e9254c23a40173312f865ac1)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:57.762010 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb7dfc-7695-499b-9795-807b32b92a39 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:57.890473 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-75eb7dfc-7695-499b-9795-807b32b92a39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99d6b751e9254c23a40173312f865ac1', 'request_id': 'req-75eb7dfc-7695-499b-9795-807b32b92a39'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:57.912576 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-75eb7dfc-7695-499b-9795-807b32b92a39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99d6b751e9254c23a40173312f865ac1', 'request_id': 'req-75eb7dfc-7695-499b-9795-807b32b92a39'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:57.928822 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-75eb7dfc-7695-499b-9795-807b32b92a39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99d6b751e9254c23a40173312f865ac1', 'request_id': 'req-75eb7dfc-7695-499b-9795-807b32b92a39'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:57.952416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-75eb7dfc-7695-499b-9795-807b32b92a39 admin admin] The token cache is being invalidate because project 99d6b751e9254c23a40173312f865ac1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:08:57.953163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-75eb7dfc-7695-499b-9795-807b32b92a39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:57.956860 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-75eb7dfc-7695-499b-9795-807b32b92a39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:57.965342 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8b47e-93c5-4d1a-a5d2-b8efcc508eb0 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:57.965631 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8b47e-93c5-4d1a-a5d2-b8efcc508eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:57.966326 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8b47e-93c5-4d1a-a5d2-b8efcc508eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:57.970901 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 891/1745] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:08:57 2022] DELETE /identity/v3/projects/99d6b751e9254c23a40173312f865ac1 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:57.994551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-03c8b47e-93c5-4d1a-a5d2-b8efcc508eb0 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:57.997671 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-03c8b47e-93c5-4d1a-a5d2-b8efcc508eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:58.021988 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:58.104663 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e487e8b-f005-426b-851a-214a4bb3517e None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:58.105249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e487e8b-f005-426b-851a-214a4bb3517e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:58.105440 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 855/1746] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:58.105945 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e487e8b-f005-426b-851a-214a4bb3517e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:58.120733 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:58.169614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-2e487e8b-f005-426b-851a-214a4bb3517e None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:58.170903 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-2e487e8b-f005-426b-851a-214a4bb3517e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:58.215246 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 892/1747] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:58.219422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8b47e-93c5-4d1a-a5d2-b8efcc508eb0 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:58.254361 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8b47e-93c5-4d1a-a5d2-b8efcc508eb0 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:58.264287 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560233d9-5449-460e-b47e-17f212835b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:58.265075 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560233d9-5449-460e-b47e-17f212835b3f admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:58.265315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560233d9-5449-460e-b47e-17f212835b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:58.265638 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560233d9-5449-460e-b47e-17f212835b3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:58.266370 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560233d9-5449-460e-b47e-17f212835b3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:08:58.266811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560233d9-5449-460e-b47e-17f212835b3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:58.269571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560233d9-5449-460e-b47e-17f212835b3f admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:58.306274 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e487e8b-f005-426b-851a-214a4bb3517e None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:58.306568 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 856/1748] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:08:58.324677 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e487e8b-f005-426b-851a-214a4bb3517e None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:58.359704 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d39cc3-cdc6-49c2-b2a3-c612489d6e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:58.360617 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d39cc3-cdc6-49c2-b2a3-c612489d6e12 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:58.361574 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d39cc3-cdc6-49c2-b2a3-c612489d6e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:58.362037 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d39cc3-cdc6-49c2-b2a3-c612489d6e12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:58.362869 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d39cc3-cdc6-49c2-b2a3-c612489d6e12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:58.365262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d39cc3-cdc6-49c2-b2a3-c612489d6e12 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:58.420085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 893/1749] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:08:58 2022] POST /identity/v3/projects => generated 393 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:58.426281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560233d9-5449-460e-b47e-17f212835b3f admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:58.452662 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560233d9-5449-460e-b47e-17f212835b3f admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:58.582410 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ead03-3b1f-4149-8fad-f21fb8333c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:58.583296 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ead03-3b1f-4149-8fad-f21fb8333c3e admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:58.583580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ead03-3b1f-4149-8fad-f21fb8333c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:58.583926 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ead03-3b1f-4149-8fad-f21fb8333c3e admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:58.584577 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ead03-3b1f-4149-8fad-f21fb8333c3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:58.586524 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ead03-3b1f-4149-8fad-f21fb8333c3e admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:58.651104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 857/1750] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:08:58 2022] POST /identity/v3/users => generated 297 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:58.654590 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d39cc3-cdc6-49c2-b2a3-c612489d6e12 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:58.671730 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d39cc3-cdc6-49c2-b2a3-c612489d6e12 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:58.682885 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694a81a-5fae-4a4c-bb43-28b0f05cae6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:58.683833 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694a81a-5fae-4a4c-bb43-28b0f05cae6f admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:58.684321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694a81a-5fae-4a4c-bb43-28b0f05cae6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:58.684811 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694a81a-5fae-4a4c-bb43-28b0f05cae6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:58.685689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694a81a-5fae-4a4c-bb43-28b0f05cae6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:58.696806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9694a81a-5fae-4a4c-bb43-28b0f05cae6f admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:58.717494 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 894/1751] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:58.722264 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ead03-3b1f-4149-8fad-f21fb8333c3e admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:58.738040 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ead03-3b1f-4149-8fad-f21fb8333c3e admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:58.757701 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e99eb16-6055-458b-8c5b-01ee6071034c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:58.758396 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99eb16-6055-458b-8c5b-01ee6071034c admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:58.758680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99eb16-6055-458b-8c5b-01ee6071034c admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:58.758955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e99eb16-6055-458b-8c5b-01ee6071034c admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:58.767619 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99eb16-6055-458b-8c5b-01ee6071034c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:58.772362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e99eb16-6055-458b-8c5b-01ee6071034c admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:58.819299 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 858/1752] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:08:58.820492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694a81a-5fae-4a4c-bb43-28b0f05cae6f admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:58.836725 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694a81a-5fae-4a4c-bb43-28b0f05cae6f admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:58.844642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe7e26-19f7-4508-8c36-7cb8645d22a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:58.855248 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe7e26-19f7-4508-8c36-7cb8645d22a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:58.855248 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe7e26-19f7-4508-8c36-7cb8645d22a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:58.855248 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe7e26-19f7-4508-8c36-7cb8645d22a4 admin admin] PATH_INFO: `/v3/projects/1b21a798cdfb45d68573741d9b241ec5/users/ed7474605fbe4d99bc08350f7d562e66/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:58.901167 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe7e26-19f7-4508-8c36-7cb8645d22a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b21a798cdfb45d68573741d9b241ec5, user_id=ed7474605fbe4d99bc08350f7d562e66, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:58.910524 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe7e26-19f7-4508-8c36-7cb8645d22a4 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:58.946962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 895/1753] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:08:58 2022] PUT /identity/v3/projects/1b21a798cdfb45d68573741d9b241ec5/users/ed7474605fbe4d99bc08350f7d562e66/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:58.957523 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa41834-fd58-4ffb-adcd-cde2cf9e37cb None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:58.958138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa41834-fd58-4ffb-adcd-cde2cf9e37cb None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:58.958601 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa41834-fd58-4ffb-adcd-cde2cf9e37cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:59.047481 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-efa41834-fd58-4ffb-adcd-cde2cf9e37cb None None] MFA Rules not processed for user `ed7474605fbe4d99bc08350f7d562e66`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:59.049924 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-efa41834-fd58-4ffb-adcd-cde2cf9e37cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:59.266358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 859/1754] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:08:59.270966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe7e26-19f7-4508-8c36-7cb8645d22a4 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:59.276406 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa41834-fd58-4ffb-adcd-cde2cf9e37cb None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:59.278125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-efa41834-fd58-4ffb-adcd-cde2cf9e37cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:59.322252 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:08:59.470920 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa41834-fd58-4ffb-adcd-cde2cf9e37cb None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:59.489194 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e96076-dd35-4dbe-85b4-b83378424632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:59.489865 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e96076-dd35-4dbe-85b4-b83378424632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:59.490150 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e96076-dd35-4dbe-85b4-b83378424632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:59.490432 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e96076-dd35-4dbe-85b4-b83378424632 admin admin] PATH_INFO: `/v3/projects/8418a7ff66e04563bc94f73b64c6af8f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:59.492117 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e96076-dd35-4dbe-85b4-b83378424632 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8418a7ff66e04563bc94f73b64c6af8f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:59.494360 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e96076-dd35-4dbe-85b4-b83378424632 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:59.550228 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe7e26-19f7-4508-8c36-7cb8645d22a4 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:08:59.564352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d7c13-7a02-4926-9473-f057aed0fd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:08:59.569648 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d7c13-7a02-4926-9473-f057aed0fd86 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:59.569648 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d7c13-7a02-4926-9473-f057aed0fd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:59.569648 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d7c13-7a02-4926-9473-f057aed0fd86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:59.569648 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d7c13-7a02-4926-9473-f057aed0fd86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:08:59.573506 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d7c13-7a02-4926-9473-f057aed0fd86 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:08:59.652340 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 896/1755] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:08:59 2022] POST /identity/v3/projects => generated 393 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:08:59.654110 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c3eea-b89a-41bc-bf7b-d53011f41429 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:08:59.654603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c3eea-b89a-41bc-bf7b-d53011f41429 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:08:59.656066 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c3eea-b89a-41bc-bf7b-d53011f41429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:08:59.679842 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c6e96076-dd35-4dbe-85b4-b83378424632 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8418a7ff66e04563bc94f73b64c6af8f', 'request_id': 'req-c6e96076-dd35-4dbe-85b4-b83378424632'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:59.695657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c6e96076-dd35-4dbe-85b4-b83378424632 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8418a7ff66e04563bc94f73b64c6af8f', 'request_id': 'req-c6e96076-dd35-4dbe-85b4-b83378424632'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:59.707813 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c6e96076-dd35-4dbe-85b4-b83378424632 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8418a7ff66e04563bc94f73b64c6af8f', 'request_id': 'req-c6e96076-dd35-4dbe-85b4-b83378424632'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:59.754178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-049c3eea-b89a-41bc-bf7b-d53011f41429 None None] MFA Rules not processed for user `10fcb673c54e484a88a0dabe7b8edb15`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:08:59.756978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-049c3eea-b89a-41bc-bf7b-d53011f41429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:59.763514 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c6e96076-dd35-4dbe-85b4-b83378424632 admin admin] The token cache is being invalidate because project 8418a7ff66e04563bc94f73b64c6af8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:08:59.763821 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c6e96076-dd35-4dbe-85b4-b83378424632 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:59.765026 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c6e96076-dd35-4dbe-85b4-b83378424632 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:08:59.772571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 860/1756] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:08:59 2022] DELETE /identity/v3/projects/8418a7ff66e04563bc94f73b64c6af8f => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:08:59.772571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e96076-dd35-4dbe-85b4-b83378424632 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:08:59.777912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c6e96076-dd35-4dbe-85b4-b83378424632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:08:59.992135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e96076-dd35-4dbe-85b4-b83378424632 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:00.020313 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 897/1757] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:08:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 367 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:00.022178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bfaa5-da35-4be6-886e-923cb170e9f1 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:00.022475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bfaa5-da35-4be6-886e-923cb170e9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:00.022754 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600bfaa5-da35-4be6-886e-923cb170e9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:00.080827 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-600bfaa5-da35-4be6-886e-923cb170e9f1 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:00.082308 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-600bfaa5-da35-4be6-886e-923cb170e9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:00.140725 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:00.146301 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:00.185730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a7f98-f741-4c8d-8a88-5192f9da1feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:00.186481 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a7f98-f741-4c8d-8a88-5192f9da1feb admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:00.186862 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a7f98-f741-4c8d-8a88-5192f9da1feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:00.187257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a7f98-f741-4c8d-8a88-5192f9da1feb admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:00.191497 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a7f98-f741-4c8d-8a88-5192f9da1feb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:00.193873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a7f98-f741-4c8d-8a88-5192f9da1feb admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:00.265146 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 861/1758] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:08:59 2022] POST /identity/v3/users => generated 303 bytes in 493 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:09:00.265146 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a7f98-f741-4c8d-8a88-5192f9da1feb admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:00.268870 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-cd5a7f98-f741-4c8d-8a88-5192f9da1feb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:00.307881 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a7f98-f741-4c8d-8a88-5192f9da1feb admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:00.317028 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455117b4-e0ea-4927-8e07-e5f61203bd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:00.317534 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455117b4-e0ea-4927-8e07-e5f61203bd9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:00.317817 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455117b4-e0ea-4927-8e07-e5f61203bd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:00.318085 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455117b4-e0ea-4927-8e07-e5f61203bd9f admin admin] PATH_INFO: `/v3/projects/d405b5c6dec74ddd85a379135a549d7b` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:00.326207 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455117b4-e0ea-4927-8e07-e5f61203bd9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d405b5c6dec74ddd85a379135a549d7b)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:00.328214 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455117b4-e0ea-4927-8e07-e5f61203bd9f admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:00.328536 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 898/1759] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:00.330360 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600bfaa5-da35-4be6-886e-923cb170e9f1 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:00.364019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600bfaa5-da35-4be6-886e-923cb170e9f1 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:00.371833 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d0ad8-afa1-4738-a9ce-f1dd1f8c3cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:00.372471 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d0ad8-afa1-4738-a9ce-f1dd1f8c3cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:00.372752 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d0ad8-afa1-4738-a9ce-f1dd1f8c3cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:00.373023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d0ad8-afa1-4738-a9ce-f1dd1f8c3cf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:00.373771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d0ad8-afa1-4738-a9ce-f1dd1f8c3cf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:00.375805 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d0ad8-afa1-4738-a9ce-f1dd1f8c3cf4 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:00.401520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 899/1760] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:09:00.416326 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d835de-44ca-4af8-bf57-12609e161211 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:00.418094 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d835de-44ca-4af8-bf57-12609e161211 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:00.418496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d835de-44ca-4af8-bf57-12609e161211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:00.457507 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-16d835de-44ca-4af8-bf57-12609e161211 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:00.457928 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-455117b4-e0ea-4927-8e07-e5f61203bd9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd405b5c6dec74ddd85a379135a549d7b', 'request_id': 'req-455117b4-e0ea-4927-8e07-e5f61203bd9f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:00.459669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-16d835de-44ca-4af8-bf57-12609e161211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:00.469676 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-455117b4-e0ea-4927-8e07-e5f61203bd9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd405b5c6dec74ddd85a379135a549d7b', 'request_id': 'req-455117b4-e0ea-4927-8e07-e5f61203bd9f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:00.516645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-455117b4-e0ea-4927-8e07-e5f61203bd9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd405b5c6dec74ddd85a379135a549d7b', 'request_id': 'req-455117b4-e0ea-4927-8e07-e5f61203bd9f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:00.547026 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-455117b4-e0ea-4927-8e07-e5f61203bd9f admin admin] The token cache is being invalidate because project d405b5c6dec74ddd85a379135a549d7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:09:00.547774 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-455117b4-e0ea-4927-8e07-e5f61203bd9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:00.549440 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-455117b4-e0ea-4927-8e07-e5f61203bd9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:00.564206 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 862/1761] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:09:00 2022] DELETE /identity/v3/projects/d405b5c6dec74ddd85a379135a549d7b => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:00.573499 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455117b4-e0ea-4927-8e07-e5f61203bd9f admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:00.576779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-455117b4-e0ea-4927-8e07-e5f61203bd9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:00.631591 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 900/1762] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:00.632175 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d835de-44ca-4af8-bf57-12609e161211 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:00.640696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-16d835de-44ca-4af8-bf57-12609e161211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:00.659631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d835de-44ca-4af8-bf57-12609e161211 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:00.671908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345329af-5ba4-4f60-9491-58ac40b7abdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:00.672619 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345329af-5ba4-4f60-9491-58ac40b7abdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:00.672865 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345329af-5ba4-4f60-9491-58ac40b7abdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:00.678229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345329af-5ba4-4f60-9491-58ac40b7abdb admin admin] PATH_INFO: `/v3/users/1015557945fb4a539a27b1b8fffc8b4c` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:00.684517 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345329af-5ba4-4f60-9491-58ac40b7abdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1015557945fb4a539a27b1b8fffc8b4c)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:00.690794 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345329af-5ba4-4f60-9491-58ac40b7abdb admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:00.700073 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455117b4-e0ea-4927-8e07-e5f61203bd9f admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:00.711004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41dad5f-79d0-4aec-897a-4dee3b2052f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:00.712902 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41dad5f-79d0-4aec-897a-4dee3b2052f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:00.713493 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41dad5f-79d0-4aec-897a-4dee3b2052f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:00.713988 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41dad5f-79d0-4aec-897a-4dee3b2052f1 admin admin] PATH_INFO: `/v3/projects/a59890050589479696934e66c2fbc97a/users/69a1b92c967d4827b5ab372ac12f20bf/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:00.772688 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41dad5f-79d0-4aec-897a-4dee3b2052f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a59890050589479696934e66c2fbc97a, user_id=69a1b92c967d4827b5ab372ac12f20bf, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:00.775825 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41dad5f-79d0-4aec-897a-4dee3b2052f1 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:00.815634 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 863/1763] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:09:00 2022] PUT /identity/v3/projects/a59890050589479696934e66c2fbc97a/users/69a1b92c967d4827b5ab372ac12f20bf/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:00.825293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41dad5f-79d0-4aec-897a-4dee3b2052f1 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:00.838086 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-345329af-5ba4-4f60-9491-58ac40b7abdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1015557945fb4a539a27b1b8fffc8b4c'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:00.855138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-345329af-5ba4-4f60-9491-58ac40b7abdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1015557945fb4a539a27b1b8fffc8b4c'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:00.871700 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-345329af-5ba4-4f60-9491-58ac40b7abdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1015557945fb4a539a27b1b8fffc8b4c'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:00.872641 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-345329af-5ba4-4f60-9491-58ac40b7abdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1015557945fb4a539a27b1b8fffc8b4c'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:00.873793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-345329af-5ba4-4f60-9491-58ac40b7abdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1015557945fb4a539a27b1b8fffc8b4c'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:00.940253 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 901/1764] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:09:00 2022] DELETE /identity/v3/users/1015557945fb4a539a27b1b8fffc8b4c => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:00.943159 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345329af-5ba4-4f60-9491-58ac40b7abdb admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:00.948969 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-345329af-5ba4-4f60-9491-58ac40b7abdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:01.030103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:01.090164 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41dad5f-79d0-4aec-897a-4dee3b2052f1 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:01.127815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:01.259601 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345329af-5ba4-4f60-9491-58ac40b7abdb admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:01.269668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c801c64-a014-4900-aa18-8f86eca5b295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:01.270653 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c801c64-a014-4900-aa18-8f86eca5b295 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:01.271018 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c801c64-a014-4900-aa18-8f86eca5b295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:01.271281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c801c64-a014-4900-aa18-8f86eca5b295 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:01.271778 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c801c64-a014-4900-aa18-8f86eca5b295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:01.273400 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c801c64-a014-4900-aa18-8f86eca5b295 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:01.286190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785aae2e-50b9-4339-93a5-9f22ee9b6cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:01.286911 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785aae2e-50b9-4339-93a5-9f22ee9b6cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:01.288309 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785aae2e-50b9-4339-93a5-9f22ee9b6cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:01.288309 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785aae2e-50b9-4339-93a5-9f22ee9b6cb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:01.289042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785aae2e-50b9-4339-93a5-9f22ee9b6cb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:09:01.289404 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785aae2e-50b9-4339-93a5-9f22ee9b6cb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:01.291639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785aae2e-50b9-4339-93a5-9f22ee9b6cb7 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:01.321894 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 902/1765] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:09:01.323546 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa4389-2829-4549-a26c-f40e63b782c8 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:01.323838 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa4389-2829-4549-a26c-f40e63b782c8 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:01.324132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa4389-2829-4549-a26c-f40e63b782c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:01.359113 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 864/1766] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 534 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:09:01.360085 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785aae2e-50b9-4339-93a5-9f22ee9b6cb7 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:01.383538 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-c1fa4389-2829-4549-a26c-f40e63b782c8 None None] MFA Rules not processed for user `4210f948dc7e4a5bac61d10eb6a62eee`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:01.384784 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c1fa4389-2829-4549-a26c-f40e63b782c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:01.388481 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785aae2e-50b9-4339-93a5-9f22ee9b6cb7 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:01.400941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e30e7-1eee-4b21-96a4-8ae301d1f7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:01.402035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e30e7-1eee-4b21-96a4-8ae301d1f7ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:01.402035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e30e7-1eee-4b21-96a4-8ae301d1f7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:01.402221 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e30e7-1eee-4b21-96a4-8ae301d1f7ab admin admin] PATH_INFO: `/v3/projects/a59890050589479696934e66c2fbc97a/users/69a1b92c967d4827b5ab372ac12f20bf/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:01.405149 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e30e7-1eee-4b21-96a4-8ae301d1f7ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=a59890050589479696934e66c2fbc97a, user_id=69a1b92c967d4827b5ab372ac12f20bf, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:01.412178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e30e7-1eee-4b21-96a4-8ae301d1f7ab admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:01.466197 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 865/1767] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:09:01 2022] PUT /identity/v3/projects/a59890050589479696934e66c2fbc97a/users/69a1b92c967d4827b5ab372ac12f20bf/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:01.467762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e30e7-1eee-4b21-96a4-8ae301d1f7ab admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:01.472971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ff0e30e7-1eee-4b21-96a4-8ae301d1f7ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:01.529537 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 903/1768] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:01.531333 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68193183-a518-4e5a-b2a0-44e5ceef7b20 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:01.531797 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68193183-a518-4e5a-b2a0-44e5ceef7b20 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:01.532174 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68193183-a518-4e5a-b2a0-44e5ceef7b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:01.574457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-68193183-a518-4e5a-b2a0-44e5ceef7b20 None None] MFA Rules not processed for user `69a1b92c967d4827b5ab372ac12f20bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:01.577638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-68193183-a518-4e5a-b2a0-44e5ceef7b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:01.644935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e30e7-1eee-4b21-96a4-8ae301d1f7ab admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:01.663195 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb9ba5-249a-4bc2-9054-4471a31ae7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:01.663945 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb9ba5-249a-4bc2-9054-4471a31ae7dd admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:01.664336 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb9ba5-249a-4bc2-9054-4471a31ae7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:01.664714 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb9ba5-249a-4bc2-9054-4471a31ae7dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:01.665468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb9ba5-249a-4bc2-9054-4471a31ae7dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:01.672327 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 904/1769] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:01 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:01.673734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb9ba5-249a-4bc2-9054-4471a31ae7dd admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:01.686555 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68193183-a518-4e5a-b2a0-44e5ceef7b20 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:01.694782 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-68193183-a518-4e5a-b2a0-44e5ceef7b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:01.771625 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 866/1770] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:09:01 2022] POST /identity/v3/projects => generated 385 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:09:01.774525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb9ba5-249a-4bc2-9054-4471a31ae7dd admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:01.779249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68193183-a518-4e5a-b2a0-44e5ceef7b20 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:01.802279 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:01.937413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb612f85-0abb-47cd-848c-0bc53331982d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:01.939133 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb612f85-0abb-47cd-848c-0bc53331982d service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:01.941409 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb612f85-0abb-47cd-848c-0bc53331982d service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:01.941891 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb612f85-0abb-47cd-848c-0bc53331982d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:02.021533 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb9ba5-249a-4bc2-9054-4471a31ae7dd admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:02.030016 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330b49d-8483-4580-97fd-a215d48d656b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:02.036196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330b49d-8483-4580-97fd-a215d48d656b admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:02.040106 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330b49d-8483-4580-97fd-a215d48d656b admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:02.040106 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330b49d-8483-4580-97fd-a215d48d656b admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:02.040569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330b49d-8483-4580-97fd-a215d48d656b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:02.053010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330b49d-8483-4580-97fd-a215d48d656b admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:02.087847 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:02.124004 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb612f85-0abb-47cd-848c-0bc53331982d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:02.125109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 867/1771] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:09:01 2022] POST /identity/v3/users => generated 293 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:09:02.129230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330b49d-8483-4580-97fd-a215d48d656b admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:02.135557 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb612f85-0abb-47cd-848c-0bc53331982d service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:02.152898 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330b49d-8483-4580-97fd-a215d48d656b admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:02.158511 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede59c88-2b5a-4c6a-8860-035e982ca45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:02.159289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede59c88-2b5a-4c6a-8860-035e982ca45f admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:02.159486 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede59c88-2b5a-4c6a-8860-035e982ca45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:02.159758 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede59c88-2b5a-4c6a-8860-035e982ca45f admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:02.160411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede59c88-2b5a-4c6a-8860-035e982ca45f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:02.161178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 905/1772] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:01 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:09:02.162113 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede59c88-2b5a-4c6a-8860-035e982ca45f admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:02.176603 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 868/1773] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:09:02.181754 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb612f85-0abb-47cd-848c-0bc53331982d service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:02.204671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb612f85-0abb-47cd-848c-0bc53331982d service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:02.213083 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097db22f-3d18-4e17-a482-42cd402acc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:02.213785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097db22f-3d18-4e17-a482-42cd402acc5b admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:02.215058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097db22f-3d18-4e17-a482-42cd402acc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:02.215058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097db22f-3d18-4e17-a482-42cd402acc5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:02.218273 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097db22f-3d18-4e17-a482-42cd402acc5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:02.222815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097db22f-3d18-4e17-a482-42cd402acc5b admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:02.250413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 906/1774] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:02 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:09:02.254826 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede59c88-2b5a-4c6a-8860-035e982ca45f admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:02.282847 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede59c88-2b5a-4c6a-8860-035e982ca45f admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:02.303821 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b36a9-810c-4197-9e7d-cf1cc8f143e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:02.305296 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b36a9-810c-4197-9e7d-cf1cc8f143e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:02.305886 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b36a9-810c-4197-9e7d-cf1cc8f143e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:02.306210 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b36a9-810c-4197-9e7d-cf1cc8f143e5 admin admin] PATH_INFO: `/v3/projects/0fc71f44f5684d61abec3d31d243c687/users/128d0d5c494441ae93546a7747d62db9/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:02.366567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097db22f-3d18-4e17-a482-42cd402acc5b admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:02.370225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-097db22f-3d18-4e17-a482-42cd402acc5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:02.398204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b36a9-810c-4197-9e7d-cf1cc8f143e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fc71f44f5684d61abec3d31d243c687, user_id=128d0d5c494441ae93546a7747d62db9, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:02.406695 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097db22f-3d18-4e17-a482-42cd402acc5b admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:02.418823 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b36a9-810c-4197-9e7d-cf1cc8f143e5 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:02.436182 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2559a6a8-0354-4053-a29b-8061bdb62f0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:02.437349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2559a6a8-0354-4053-a29b-8061bdb62f0a service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:02.437978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2559a6a8-0354-4053-a29b-8061bdb62f0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:02.438333 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2559a6a8-0354-4053-a29b-8061bdb62f0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:02.441295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-2559a6a8-0354-4053-a29b-8061bdb62f0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:02.483171 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 869/1775] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:09:02 2022] PUT /identity/v3/projects/0fc71f44f5684d61abec3d31d243c687/users/128d0d5c494441ae93546a7747d62db9/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:02.484695 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5991e-84e1-4d86-8cea-705cbe089e5d None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:02.485011 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5991e-84e1-4d86-8cea-705cbe089e5d None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:02.485258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5991e-84e1-4d86-8cea-705cbe089e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:02.527537 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-a7c5991e-84e1-4d86-8cea-705cbe089e5d None None] MFA Rules not processed for user `07a825fddbc04f918bcd631976038cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:02.529098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-a7c5991e-84e1-4d86-8cea-705cbe089e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:02.613513 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2559a6a8-0354-4053-a29b-8061bdb62f0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:02.618469 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2559a6a8-0354-4053-a29b-8061bdb62f0a service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:02.640160 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 870/1776] 10.222.0.18 () {58 vars in 1015 bytes} [Fri Sep 2 09:09:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:02.641432 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5991e-84e1-4d86-8cea-705cbe089e5d None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:02.654813 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 907/1777] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:02 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:09:02.659450 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-a7c5991e-84e1-4d86-8cea-705cbe089e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:02.663638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428e58b-89ef-45a4-8047-bd7d84106975 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:02.663932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428e58b-89ef-45a4-8047-bd7d84106975 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:02.664214 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428e58b-89ef-45a4-8047-bd7d84106975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:02.756476 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-8428e58b-89ef-45a4-8047-bd7d84106975 None None] MFA Rules not processed for user `128d0d5c494441ae93546a7747d62db9`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:02.760803 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8428e58b-89ef-45a4-8047-bd7d84106975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:02.922357 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:02.956203 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5991e-84e1-4d86-8cea-705cbe089e5d None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:02.969997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6aedd5-7eb2-4f3f-96d9-4441cc632a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:02.971361 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aedd5-7eb2-4f3f-96d9-4441cc632a16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:02.971645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aedd5-7eb2-4f3f-96d9-4441cc632a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:02.973046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6aedd5-7eb2-4f3f-96d9-4441cc632a16 admin admin] PATH_INFO: `/v3/projects/eaef6c371133417e913918b0bbce9f48` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:02.977352 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6aedd5-7eb2-4f3f-96d9-4441cc632a16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaef6c371133417e913918b0bbce9f48)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:02.983182 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6aedd5-7eb2-4f3f-96d9-4441cc632a16 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:03.008781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 908/1778] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:03.012313 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428e58b-89ef-45a4-8047-bd7d84106975 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:03.019088 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8428e58b-89ef-45a4-8047-bd7d84106975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:03.053911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428e58b-89ef-45a4-8047-bd7d84106975 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:03.066202 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588dacc-d811-445c-bd0b-00401f64619f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:03.067655 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588dacc-d811-445c-bd0b-00401f64619f service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:03.069170 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588dacc-d811-445c-bd0b-00401f64619f service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:03.071553 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588dacc-d811-445c-bd0b-00401f64619f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:03.095905 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a588dacc-d811-445c-bd0b-00401f64619f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:03.097781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a588dacc-d811-445c-bd0b-00401f64619f service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:03.114428 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-ec6aedd5-7eb2-4f3f-96d9-4441cc632a16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaef6c371133417e913918b0bbce9f48', 'request_id': 'req-ec6aedd5-7eb2-4f3f-96d9-4441cc632a16'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:03.124695 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-ec6aedd5-7eb2-4f3f-96d9-4441cc632a16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaef6c371133417e913918b0bbce9f48', 'request_id': 'req-ec6aedd5-7eb2-4f3f-96d9-4441cc632a16'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:03.136428 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-ec6aedd5-7eb2-4f3f-96d9-4441cc632a16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaef6c371133417e913918b0bbce9f48', 'request_id': 'req-ec6aedd5-7eb2-4f3f-96d9-4441cc632a16'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:03.142324 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:03.151662 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 909/1779] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:09:03.159045 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588dacc-d811-445c-bd0b-00401f64619f service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:03.163384 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a588dacc-d811-445c-bd0b-00401f64619f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:03.178159 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-ec6aedd5-7eb2-4f3f-96d9-4441cc632a16 admin admin] The token cache is being invalidate because project eaef6c371133417e913918b0bbce9f48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:09:03.178602 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-ec6aedd5-7eb2-4f3f-96d9-4441cc632a16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:03.181394 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-ec6aedd5-7eb2-4f3f-96d9-4441cc632a16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:03.185093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 871/1780] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:09:02 2022] DELETE /identity/v3/projects/eaef6c371133417e913918b0bbce9f48 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:03.186112 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6aedd5-7eb2-4f3f-96d9-4441cc632a16 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:03.192348 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ec6aedd5-7eb2-4f3f-96d9-4441cc632a16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:03.228560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588dacc-d811-445c-bd0b-00401f64619f service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:03.353436 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae87fb91-5bcf-469b-960f-da6d1fbaa238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:03.354186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87fb91-5bcf-469b-960f-da6d1fbaa238 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:03.359182 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87fb91-5bcf-469b-960f-da6d1fbaa238 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:03.359182 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae87fb91-5bcf-469b-960f-da6d1fbaa238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:03.382988 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-ae87fb91-5bcf-469b-960f-da6d1fbaa238 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:03.393673 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6aedd5-7eb2-4f3f-96d9-4441cc632a16 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:03.406352 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c2d4d-b1d1-4578-9b13-e631b0a55535 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:03.406990 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c2d4d-b1d1-4578-9b13-e631b0a55535 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:03.421151 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c2d4d-b1d1-4578-9b13-e631b0a55535 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:03.421527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c2d4d-b1d1-4578-9b13-e631b0a55535 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:03.425688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae87fb91-5bcf-469b-960f-da6d1fbaa238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:03.427497 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae87fb91-5bcf-469b-960f-da6d1fbaa238 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:03.436250 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-973c2d4d-b1d1-4578-9b13-e631b0a55535 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:03.447603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 910/1781] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:09:03.450577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97ff06-d362-4ced-b49b-24f9d0b582b9 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:03.450577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97ff06-d362-4ced-b49b-24f9d0b582b9 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:03.450778 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c97ff06-d362-4ced-b49b-24f9d0b582b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:03.528953 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-9c97ff06-d362-4ced-b49b-24f9d0b582b9 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:03.530448 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-9c97ff06-d362-4ced-b49b-24f9d0b582b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:03.621657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c2d4d-b1d1-4578-9b13-e631b0a55535 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:03.639793 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c2d4d-b1d1-4578-9b13-e631b0a55535 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:03.678283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 872/1782] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:09:03.691105 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 911/1783] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:03.701474 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026da7e4-446c-421a-888e-ca5d1a985518 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:03.701815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026da7e4-446c-421a-888e-ca5d1a985518 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:03.702007 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026da7e4-446c-421a-888e-ca5d1a985518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:03.743325 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97ff06-d362-4ced-b49b-24f9d0b582b9 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:03.745688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-9c97ff06-d362-4ced-b49b-24f9d0b582b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:03.774864 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c97ff06-d362-4ced-b49b-24f9d0b582b9 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:03.776062 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-026da7e4-446c-421a-888e-ca5d1a985518 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:03.777298 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-026da7e4-446c-421a-888e-ca5d1a985518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:03.789474 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ef6c5-8709-4d0c-b6a2-f5cf2dce765f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:03.789953 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 873/1784] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:03.790226 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ef6c5-8709-4d0c-b6a2-f5cf2dce765f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:03.790535 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ef6c5-8709-4d0c-b6a2-f5cf2dce765f admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:03.790886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ef6c5-8709-4d0c-b6a2-f5cf2dce765f admin admin] PATH_INFO: `/v3/users/128d0d5c494441ae93546a7747d62db9` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:03.793930 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ef6c5-8709-4d0c-b6a2-f5cf2dce765f admin admin] RBAC: Authorizing `identity:delete_user(user_id=128d0d5c494441ae93546a7747d62db9)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:03.794445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026da7e4-446c-421a-888e-ca5d1a985518 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:03.797186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ef6c5-8709-4d0c-b6a2-f5cf2dce765f admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:03.806632 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026da7e4-446c-421a-888e-ca5d1a985518 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:03.828391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ab78f-a53e-49d1-b543-2da0caf3e6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:03.829107 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ab78f-a53e-49d1-b543-2da0caf3e6a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:03.829386 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ab78f-a53e-49d1-b543-2da0caf3e6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:03.829707 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ab78f-a53e-49d1-b543-2da0caf3e6a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:03.830444 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ab78f-a53e-49d1-b543-2da0caf3e6a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:09:03.830702 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ab78f-a53e-49d1-b543-2da0caf3e6a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:03.835005 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ab78f-a53e-49d1-b543-2da0caf3e6a5 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:03.880267 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 874/1785] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:09:03.881408 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ab78f-a53e-49d1-b543-2da0caf3e6a5 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:03.905539 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ab78f-a53e-49d1-b543-2da0caf3e6a5 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:03.912140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe81e9f-fed6-44f6-95de-12bcd9ae1f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:03.913040 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe81e9f-fed6-44f6-95de-12bcd9ae1f59 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:03.913526 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe81e9f-fed6-44f6-95de-12bcd9ae1f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:03.914117 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe81e9f-fed6-44f6-95de-12bcd9ae1f59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:03.915738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe81e9f-fed6-44f6-95de-12bcd9ae1f59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:03.918726 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe81e9f-fed6-44f6-95de-12bcd9ae1f59 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:03.943525 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-541ef6c5-8709-4d0c-b6a2-f5cf2dce765f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '128d0d5c494441ae93546a7747d62db9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:03.956239 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-541ef6c5-8709-4d0c-b6a2-f5cf2dce765f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '128d0d5c494441ae93546a7747d62db9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:03.975468 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-541ef6c5-8709-4d0c-b6a2-f5cf2dce765f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '128d0d5c494441ae93546a7747d62db9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:03.976659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-541ef6c5-8709-4d0c-b6a2-f5cf2dce765f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '128d0d5c494441ae93546a7747d62db9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:03.977581 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-541ef6c5-8709-4d0c-b6a2-f5cf2dce765f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '128d0d5c494441ae93546a7747d62db9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:03.980114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:04.002752 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 875/1786] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:09:03 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:09:04.005873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe81e9f-fed6-44f6-95de-12bcd9ae1f59 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:04.010157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-dbe81e9f-fed6-44f6-95de-12bcd9ae1f59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:04.028917 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 912/1787] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:09:03 2022] DELETE /identity/v3/users/128d0d5c494441ae93546a7747d62db9 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:04.037184 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49a045-7d6a-4b54-904a-f93fea5636f6 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:04.037184 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49a045-7d6a-4b54-904a-f93fea5636f6 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:04.037718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49a045-7d6a-4b54-904a-f93fea5636f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:04.105865 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-0b49a045-7d6a-4b54-904a-f93fea5636f6 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:04.107316 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0b49a045-7d6a-4b54-904a-f93fea5636f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:04.270355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe81e9f-fed6-44f6-95de-12bcd9ae1f59 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:04.312344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:04.361220 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 913/1788] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:04.458433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bee812-7af7-4520-8543-6a9338578ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:04.463593 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bee812-7af7-4520-8543-6a9338578ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:04.464156 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bee812-7af7-4520-8543-6a9338578ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:04.464636 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bee812-7af7-4520-8543-6a9338578ed5 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:04.466528 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bee812-7af7-4520-8543-6a9338578ed5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:04.477534 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bee812-7af7-4520-8543-6a9338578ed5 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:04.486617 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49a045-7d6a-4b54-904a-f93fea5636f6 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:04.488231 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0b49a045-7d6a-4b54-904a-f93fea5636f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:04.571954 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 876/1789] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:09:04 2022] POST /identity/v3/users => generated 287 bytes in 568 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:09:04.578318 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bee812-7af7-4520-8543-6a9338578ed5 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:04.608133 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bee812-7af7-4520-8543-6a9338578ed5 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:04.622125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c131a-2113-44b8-985c-6a0916f08598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:04.622818 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c131a-2113-44b8-985c-6a0916f08598 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:04.623960 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c131a-2113-44b8-985c-6a0916f08598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:04.624252 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c131a-2113-44b8-985c-6a0916f08598 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:04.626880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c131a-2113-44b8-985c-6a0916f08598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:04.631975 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c131a-2113-44b8-985c-6a0916f08598 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:04.669288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 877/1790] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:09:04.670398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c131a-2113-44b8-985c-6a0916f08598 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:04.704793 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c131a-2113-44b8-985c-6a0916f08598 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:04.727238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca2c0e-d2d7-4630-8147-bc4b69c23ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:04.731038 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca2c0e-d2d7-4630-8147-bc4b69c23ecf admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:04.732031 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca2c0e-d2d7-4630-8147-bc4b69c23ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:04.732354 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49a045-7d6a-4b54-904a-f93fea5636f6 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:04.734119 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca2c0e-d2d7-4630-8147-bc4b69c23ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:04.737384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca2c0e-d2d7-4630-8147-bc4b69c23ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:04.744158 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca2c0e-d2d7-4630-8147-bc4b69c23ecf admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:04.745295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e753493-463a-4ec7-a568-c7f6ef0f1842 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:04.745932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e753493-463a-4ec7-a568-c7f6ef0f1842 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:04.746213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e753493-463a-4ec7-a568-c7f6ef0f1842 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:04.746483 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e753493-463a-4ec7-a568-c7f6ef0f1842 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:04.805041 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 878/1791] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:09:04.805456 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e753493-463a-4ec7-a568-c7f6ef0f1842 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:04.806101 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e753493-463a-4ec7-a568-c7f6ef0f1842 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:04.814467 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca2c0e-d2d7-4630-8147-bc4b69c23ecf admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:04.835628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 914/1792] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:09:04.859146 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca2c0e-d2d7-4630-8147-bc4b69c23ecf admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:04.883213 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e15f2-d704-4a77-b54b-dcb311aa53c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:04.883901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e15f2-d704-4a77-b54b-dcb311aa53c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:04.884893 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e15f2-d704-4a77-b54b-dcb311aa53c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:04.884893 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3e15f2-d704-4a77-b54b-dcb311aa53c2 admin admin] PATH_INFO: `/v3/projects/34750dac24ce4283a8f9f8697773bbe7/users/5b755e93453c4c2594afe759f6c4bf0f/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:04.959979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e15f2-d704-4a77-b54b-dcb311aa53c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34750dac24ce4283a8f9f8697773bbe7, user_id=5b755e93453c4c2594afe759f6c4bf0f, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:04.962662 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3e15f2-d704-4a77-b54b-dcb311aa53c2 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:05.018399 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 879/1793] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:09:04 2022] PUT /identity/v3/projects/34750dac24ce4283a8f9f8697773bbe7/users/5b755e93453c4c2594afe759f6c4bf0f/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:05.029634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b1c8d-f4e2-4b52-9b5d-84602b375bd2 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:05.029817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b1c8d-f4e2-4b52-9b5d-84602b375bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:05.030517 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1b1c8d-f4e2-4b52-9b5d-84602b375bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:05.139097 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-7a1b1c8d-f4e2-4b52-9b5d-84602b375bd2 None None] MFA Rules not processed for user `5b755e93453c4c2594afe759f6c4bf0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:05.153479 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-7a1b1c8d-f4e2-4b52-9b5d-84602b375bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:05.371125 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 915/1794] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:05.375274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e15f2-d704-4a77-b54b-dcb311aa53c2 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:05.484876 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c8e24-99cc-4ee6-b019-818a9904f93d None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:05.485573 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c8e24-99cc-4ee6-b019-818a9904f93d None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:05.485898 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c8e24-99cc-4ee6-b019-818a9904f93d None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:05.488960 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 916/1795] 10.222.0.36 () {54 vars in 921 bytes} [Fri Sep 2 09:09:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:09:05.545819 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3e15f2-d704-4a77-b54b-dcb311aa53c2 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:05.546083 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20b35f-457a-4062-86d5-ed1a05c08c6c None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:05.546403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20b35f-457a-4062-86d5-ed1a05c08c6c None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:05.546697 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20b35f-457a-4062-86d5-ed1a05c08c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:05.562413 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae6016-2ef8-4ebf-b93e-800333b605ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:05.563058 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae6016-2ef8-4ebf-b93e-800333b605ca admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:05.564876 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae6016-2ef8-4ebf-b93e-800333b605ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:05.566361 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae6016-2ef8-4ebf-b93e-800333b605ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:05.567053 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae6016-2ef8-4ebf-b93e-800333b605ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:05.583613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae6016-2ef8-4ebf-b93e-800333b605ca admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:05.603547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-0f20b35f-457a-4062-86d5-ed1a05c08c6c None None] MFA Rules not processed for user `cce502a83f5343558c2136c0193e7d85`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:05.605056 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0f20b35f-457a-4062-86d5-ed1a05c08c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:05.648673 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:05.659104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 880/1796] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:09:05 2022] POST /identity/v3/projects => generated 371 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:09:05.661450 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae6016-2ef8-4ebf-b93e-800333b605ca admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:05.798168 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 917/1797] 10.222.0.36 () {58 vars in 1018 bytes} [Fri Sep 2 09:09:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:05.859612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:05.909307 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae6016-2ef8-4ebf-b93e-800333b605ca admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:05.923718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38dc71-75e0-4937-8d21-0a50da7fd348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:05.925823 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38dc71-75e0-4937-8d21-0a50da7fd348 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:05.926349 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38dc71-75e0-4937-8d21-0a50da7fd348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:05.927410 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb38dc71-75e0-4937-8d21-0a50da7fd348 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:05.930741 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb38dc71-75e0-4937-8d21-0a50da7fd348 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:05.933990 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb38dc71-75e0-4937-8d21-0a50da7fd348 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:06.020042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20b35f-457a-4062-86d5-ed1a05c08c6c None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:06.032287 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 881/1798] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:09:05 2022] POST /identity/v3/users => generated 292 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:09:06.039429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20b35f-457a-4062-86d5-ed1a05c08c6c None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:06.050226 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166c79f-2d46-4811-8b6d-c5e0d3517ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:06.051549 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166c79f-2d46-4811-8b6d-c5e0d3517ecd admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:06.054331 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166c79f-2d46-4811-8b6d-c5e0d3517ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:06.057313 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166c79f-2d46-4811-8b6d-c5e0d3517ecd admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:06.058831 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166c79f-2d46-4811-8b6d-c5e0d3517ecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:06.074811 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166c79f-2d46-4811-8b6d-c5e0d3517ecd admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:06.121655 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 918/1799] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:09:06.125837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38dc71-75e0-4937-8d21-0a50da7fd348 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:06.182158 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb38dc71-75e0-4937-8d21-0a50da7fd348 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:06.205343 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31718e-df4a-4880-ad56-50d445eaa8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:06.207703 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31718e-df4a-4880-ad56-50d445eaa8f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:06.209403 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31718e-df4a-4880-ad56-50d445eaa8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:06.210615 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31718e-df4a-4880-ad56-50d445eaa8f1 admin admin] PATH_INFO: `/v3/projects/a987b62a9a174e3f9da5b19cc69d86de/users/920108cf870b4865843083e0c3fd2aed/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:06.258018 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31718e-df4a-4880-ad56-50d445eaa8f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a987b62a9a174e3f9da5b19cc69d86de, user_id=920108cf870b4865843083e0c3fd2aed, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:06.267858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31718e-df4a-4880-ad56-50d445eaa8f1 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:06.299040 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 882/1800] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:09:06 2022] PUT /identity/v3/projects/a987b62a9a174e3f9da5b19cc69d86de/users/920108cf870b4865843083e0c3fd2aed/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:09:06.302838 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166c79f-2d46-4811-8b6d-c5e0d3517ecd admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:06.321107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166c79f-2d46-4811-8b6d-c5e0d3517ecd admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:06.551679 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afeda82-4214-44e1-827a-6c681d27dc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:06.552318 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afeda82-4214-44e1-827a-6c681d27dc80 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:06.553186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afeda82-4214-44e1-827a-6c681d27dc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:06.553572 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afeda82-4214-44e1-827a-6c681d27dc80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:06.555685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afeda82-4214-44e1-827a-6c681d27dc80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:06.562331 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afeda82-4214-44e1-827a-6c681d27dc80 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:06.585837 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 919/1801] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:09:06.589484 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31718e-df4a-4880-ad56-50d445eaa8f1 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:06.636760 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31718e-df4a-4880-ad56-50d445eaa8f1 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:06.672324 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d19afa-7f04-4002-b5a0-bd6dc324c45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:06.674576 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d19afa-7f04-4002-b5a0-bd6dc324c45d admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:06.683537 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d19afa-7f04-4002-b5a0-bd6dc324c45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:06.684425 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d19afa-7f04-4002-b5a0-bd6dc324c45d admin admin] PATH_INFO: `/v3/projects/a987b62a9a174e3f9da5b19cc69d86de/users/920108cf870b4865843083e0c3fd2aed/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:06.690782 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d19afa-7f04-4002-b5a0-bd6dc324c45d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a987b62a9a174e3f9da5b19cc69d86de, user_id=920108cf870b4865843083e0c3fd2aed, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:06.697763 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d19afa-7f04-4002-b5a0-bd6dc324c45d admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:06.749282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 883/1802] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:09:06 2022] PUT /identity/v3/projects/a987b62a9a174e3f9da5b19cc69d86de/users/920108cf870b4865843083e0c3fd2aed/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:06.759890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f591a-fa39-4c14-a0bd-733792096324 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:06.760521 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f591a-fa39-4c14-a0bd-733792096324 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:06.761116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f591a-fa39-4c14-a0bd-733792096324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:06.827722 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-eb0f591a-fa39-4c14-a0bd-733792096324 None None] MFA Rules not processed for user `920108cf870b4865843083e0c3fd2aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:06.829984 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-eb0f591a-fa39-4c14-a0bd-733792096324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:06.918616 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 920/1803] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:06.952168 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d19afa-7f04-4002-b5a0-bd6dc324c45d admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:06.954734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-e4d19afa-7f04-4002-b5a0-bd6dc324c45d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:07.086385 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d19afa-7f04-4002-b5a0-bd6dc324c45d admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:07.096093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6433b-5935-494f-b6f7-f81c5cc62b0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:07.096620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6433b-5935-494f-b6f7-f81c5cc62b0a service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:07.097007 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6433b-5935-494f-b6f7-f81c5cc62b0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:07.098420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6433b-5935-494f-b6f7-f81c5cc62b0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:07.121139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6433b-5935-494f-b6f7-f81c5cc62b0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:07.124986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6433b-5935-494f-b6f7-f81c5cc62b0a service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:07.142423 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f591a-fa39-4c14-a0bd-733792096324 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:07.144629 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-eb0f591a-fa39-4c14-a0bd-733792096324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:07.171964 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 884/1804] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:09:07.186416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:07.287242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f591a-fa39-4c14-a0bd-733792096324 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:07.295624 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6e454-a9b5-4fea-8d00-3d4238e9fd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:07.296714 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6e454-a9b5-4fea-8d00-3d4238e9fd39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:07.297091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6e454-a9b5-4fea-8d00-3d4238e9fd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:07.297411 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6e454-a9b5-4fea-8d00-3d4238e9fd39 admin admin] PATH_INFO: `/v3/projects/0fc71f44f5684d61abec3d31d243c687` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:07.299008 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6e454-a9b5-4fea-8d00-3d4238e9fd39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fc71f44f5684d61abec3d31d243c687)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:07.301811 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6e454-a9b5-4fea-8d00-3d4238e9fd39 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:07.386960 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6433b-5935-494f-b6f7-f81c5cc62b0a service cinder] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:07.431779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-03b6e454-a9b5-4fea-8d00-3d4238e9fd39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fc71f44f5684d61abec3d31d243c687', 'request_id': 'req-03b6e454-a9b5-4fea-8d00-3d4238e9fd39'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:07.435850 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6433b-5935-494f-b6f7-f81c5cc62b0a service cinder] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:07.442005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-03b6e454-a9b5-4fea-8d00-3d4238e9fd39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fc71f44f5684d61abec3d31d243c687', 'request_id': 'req-03b6e454-a9b5-4fea-8d00-3d4238e9fd39'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:07.455310 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-03b6e454-a9b5-4fea-8d00-3d4238e9fd39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fc71f44f5684d61abec3d31d243c687', 'request_id': 'req-03b6e454-a9b5-4fea-8d00-3d4238e9fd39'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:07.473820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-03b6e454-a9b5-4fea-8d00-3d4238e9fd39 admin admin] The token cache is being invalidate because project 0fc71f44f5684d61abec3d31d243c687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:09:07.474115 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-03b6e454-a9b5-4fea-8d00-3d4238e9fd39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:07.474784 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-03b6e454-a9b5-4fea-8d00-3d4238e9fd39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:07.480613 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 921/1805] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:09:07 2022] DELETE /identity/v3/projects/0fc71f44f5684d61abec3d31d243c687 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:07.482096 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd9b21-0d76-47f1-b934-7268dc8b6f5f None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:07.482538 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd9b21-0d76-47f1-b934-7268dc8b6f5f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:07.482880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd9b21-0d76-47f1-b934-7268dc8b6f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:07.510261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-4fdd9b21-0d76-47f1-b934-7268dc8b6f5f None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:07.512637 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-4fdd9b21-0d76-47f1-b934-7268dc8b6f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:07.608386 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce877137-aaf2-4d3b-83cc-0bb9b5eab28c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:07.609108 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce877137-aaf2-4d3b-83cc-0bb9b5eab28c service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:07.609366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce877137-aaf2-4d3b-83cc-0bb9b5eab28c service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:07.609720 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce877137-aaf2-4d3b-83cc-0bb9b5eab28c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:07.618464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ce877137-aaf2-4d3b-83cc-0bb9b5eab28c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:07.671109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 922/1806] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:07.683392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa173f4-542d-487f-94b9-8955d3b200af None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:07.683684 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa173f4-542d-487f-94b9-8955d3b200af None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:07.684106 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa173f4-542d-487f-94b9-8955d3b200af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:07.717114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-afa173f4-542d-487f-94b9-8955d3b200af None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:07.722993 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-afa173f4-542d-487f-94b9-8955d3b200af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:07.750581 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 923/1807] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:07.752452 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa173f4-542d-487f-94b9-8955d3b200af None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:07.776472 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa173f4-542d-487f-94b9-8955d3b200af None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:07.792893 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c5dbd-91aa-4454-ba1b-c50c63abc558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:07.793927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c5dbd-91aa-4454-ba1b-c50c63abc558 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:07.794230 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c5dbd-91aa-4454-ba1b-c50c63abc558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:07.794692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c5dbd-91aa-4454-ba1b-c50c63abc558 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:07.795803 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c5dbd-91aa-4454-ba1b-c50c63abc558 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:09:07.796065 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c5dbd-91aa-4454-ba1b-c50c63abc558 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:07.802557 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c5dbd-91aa-4454-ba1b-c50c63abc558 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:07.826695 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:07.829706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 924/1808] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:09:07.835814 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c5dbd-91aa-4454-ba1b-c50c63abc558 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:07.852377 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce877137-aaf2-4d3b-83cc-0bb9b5eab28c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:07.857974 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c5dbd-91aa-4454-ba1b-c50c63abc558 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:07.859950 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce877137-aaf2-4d3b-83cc-0bb9b5eab28c service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:07.870284 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ee956-0c08-4e06-b64e-f7b630de39b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:07.871012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ee956-0c08-4e06-b64e-f7b630de39b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:07.871286 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ee956-0c08-4e06-b64e-f7b630de39b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:07.871642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ee956-0c08-4e06-b64e-f7b630de39b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:07.872285 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ee956-0c08-4e06-b64e-f7b630de39b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:07.877303 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ee956-0c08-4e06-b64e-f7b630de39b9 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:07.902090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 885/1809] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:09:07.932344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee545a94-6bf3-4459-acbf-0dcd56ad596c None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:07.933186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee545a94-6bf3-4459-acbf-0dcd56ad596c None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:07.933870 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee545a94-6bf3-4459-acbf-0dcd56ad596c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:08.049921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-ee545a94-6bf3-4459-acbf-0dcd56ad596c None None] MFA Rules not processed for user `e8b5ce2249a84c01ab57c03b656b2949`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:08.059365 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ee545a94-6bf3-4459-acbf-0dcd56ad596c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:08.071058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 925/1810] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:09:07 2022] POST /identity/v3/projects => generated 371 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:09:08.078945 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ee956-0c08-4e06-b64e-f7b630de39b9 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:08.114967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:08.396474 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ee956-0c08-4e06-b64e-f7b630de39b9 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:08.423736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ad59c-939b-4859-a998-d18f4d931e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:08.426424 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ad59c-939b-4859-a998-d18f4d931e6f admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:08.426895 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ad59c-939b-4859-a998-d18f4d931e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:08.428587 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ad59c-939b-4859-a998-d18f4d931e6f admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:08.429470 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ad59c-939b-4859-a998-d18f4d931e6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:08.431546 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ad59c-939b-4859-a998-d18f4d931e6f admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:08.434935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 886/1811] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 505 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:08.550615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 926/1812] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:09:08 2022] POST /identity/v3/users => generated 286 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:09:08.557921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee545a94-6bf3-4459-acbf-0dcd56ad596c None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:08.560465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ad59c-939b-4859-a998-d18f4d931e6f admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:08.561102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ee545a94-6bf3-4459-acbf-0dcd56ad596c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:08.595540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ad59c-939b-4859-a998-d18f4d931e6f admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:08.611599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627496d-a93a-464e-a28f-695fcb12bbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:08.614001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627496d-a93a-464e-a28f-695fcb12bbdf admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:08.615833 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627496d-a93a-464e-a28f-695fcb12bbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:08.618817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627496d-a93a-464e-a28f-695fcb12bbdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:08.620389 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627496d-a93a-464e-a28f-695fcb12bbdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:08.632708 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627496d-a93a-464e-a28f-695fcb12bbdf admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:08.695185 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 927/1813] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:09:08.696319 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627496d-a93a-464e-a28f-695fcb12bbdf admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:08.729138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627496d-a93a-464e-a28f-695fcb12bbdf admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:08.766088 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376bf9f-96dd-4b21-8ea8-d74654fca852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:08.781089 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376bf9f-96dd-4b21-8ea8-d74654fca852 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:08.781551 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376bf9f-96dd-4b21-8ea8-d74654fca852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:08.784756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376bf9f-96dd-4b21-8ea8-d74654fca852 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:08.788981 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376bf9f-96dd-4b21-8ea8-d74654fca852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:08.802933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376bf9f-96dd-4b21-8ea8-d74654fca852 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:08.839979 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 928/1814] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:09:08.846773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376bf9f-96dd-4b21-8ea8-d74654fca852 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:08.874155 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376bf9f-96dd-4b21-8ea8-d74654fca852 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:08.882563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463dea4-aae2-4405-ab51-c2c566f8b4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:08.883319 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463dea4-aae2-4405-ab51-c2c566f8b4be admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:08.883676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463dea4-aae2-4405-ab51-c2c566f8b4be admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:08.883995 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463dea4-aae2-4405-ab51-c2c566f8b4be admin admin] PATH_INFO: `/v3/projects/b1f68ddd905e4704b2d0b8d3beb7b2ec/users/266689c4a4864e90a3a719c52712aacf/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:08.933922 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463dea4-aae2-4405-ab51-c2c566f8b4be admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1f68ddd905e4704b2d0b8d3beb7b2ec, user_id=266689c4a4864e90a3a719c52712aacf, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:08.940813 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463dea4-aae2-4405-ab51-c2c566f8b4be admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:08.944135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee545a94-6bf3-4459-acbf-0dcd56ad596c None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:08.971295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 929/1815] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:09:08 2022] PUT /identity/v3/projects/b1f68ddd905e4704b2d0b8d3beb7b2ec/users/266689c4a4864e90a3a719c52712aacf/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:09:08.976722 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463dea4-aae2-4405-ab51-c2c566f8b4be admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:08.980204 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6463dea4-aae2-4405-ab51-c2c566f8b4be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:09.261048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463dea4-aae2-4405-ab51-c2c566f8b4be admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:09.286460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f8b6b-ad5c-4fa8-91a2-adc4d6a6f95d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:09.290571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f8b6b-ad5c-4fa8-91a2-adc4d6a6f95d service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:09.291289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f8b6b-ad5c-4fa8-91a2-adc4d6a6f95d service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:09.292040 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f8b6b-ad5c-4fa8-91a2-adc4d6a6f95d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:09.301584 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-fb2f8b6b-ad5c-4fa8-91a2-adc4d6a6f95d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:09.326580 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd3c1b9-3007-4765-8658-06b4f8b74266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:09.331188 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3c1b9-3007-4765-8658-06b4f8b74266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:09.333848 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3c1b9-3007-4765-8658-06b4f8b74266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:09.336192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd3c1b9-3007-4765-8658-06b4f8b74266 admin admin] PATH_INFO: `/v3/users/ed7474605fbe4d99bc08350f7d562e66` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:09.339386 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3c1b9-3007-4765-8658-06b4f8b74266 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed7474605fbe4d99bc08350f7d562e66)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:09.340386 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f8b6b-ad5c-4fa8-91a2-adc4d6a6f95d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:09.344911 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f8b6b-ad5c-4fa8-91a2-adc4d6a6f95d service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:09.346994 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd3c1b9-3007-4765-8658-06b4f8b74266 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:09.414336 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 887/1816] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 857 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:09:09.420689 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e6460-4a37-4ca1-a480-30cb800d51d5 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:09.423296 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e6460-4a37-4ca1-a480-30cb800d51d5 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:09.423808 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e6460-4a37-4ca1-a480-30cb800d51d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:09.545735 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-e23e6460-4a37-4ca1-a480-30cb800d51d5 None None] MFA Rules not processed for user `266689c4a4864e90a3a719c52712aacf`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:09.547139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-e23e6460-4a37-4ca1-a480-30cb800d51d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:09.583003 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-7fd3c1b9-3007-4765-8658-06b4f8b74266 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed7474605fbe4d99bc08350f7d562e66'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:09.592089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:09.654636 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:09.677084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-7fd3c1b9-3007-4765-8658-06b4f8b74266 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed7474605fbe4d99bc08350f7d562e66'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:09.730094 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-7fd3c1b9-3007-4765-8658-06b4f8b74266 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed7474605fbe4d99bc08350f7d562e66'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:09.731987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-7fd3c1b9-3007-4765-8658-06b4f8b74266 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed7474605fbe4d99bc08350f7d562e66'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:09.736583 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-7fd3c1b9-3007-4765-8658-06b4f8b74266 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed7474605fbe4d99bc08350f7d562e66'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:09.822579 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 888/1817] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:09.842291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e6460-4a37-4ca1-a480-30cb800d51d5 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:09.849395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-e23e6460-4a37-4ca1-a480-30cb800d51d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:09.914353 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 930/1818] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:09:08 2022] DELETE /identity/v3/users/ed7474605fbe4d99bc08350f7d562e66 => generated 0 bytes in 942 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:09.934709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273bef9-ddd7-4306-9843-9dbab0f85696 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:09.935064 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273bef9-ddd7-4306-9843-9dbab0f85696 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:09.937148 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273bef9-ddd7-4306-9843-9dbab0f85696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:10.010540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-2273bef9-ddd7-4306-9843-9dbab0f85696 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:10.015281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-2273bef9-ddd7-4306-9843-9dbab0f85696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:10.075458 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e6460-4a37-4ca1-a480-30cb800d51d5 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:10.200773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 931/1819] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:10.218027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a324e7f-9308-4535-8361-42ab4cc08ec9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:10.222913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a324e7f-9308-4535-8361-42ab4cc08ec9 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:10.223531 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a324e7f-9308-4535-8361-42ab4cc08ec9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:10.224035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a324e7f-9308-4535-8361-42ab4cc08ec9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:10.231037 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273bef9-ddd7-4306-9843-9dbab0f85696 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:10.233130 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-2273bef9-ddd7-4306-9843-9dbab0f85696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:10.370497 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273bef9-ddd7-4306-9843-9dbab0f85696 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:10.374634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc34af-f2d7-4d03-97eb-11da75c80bc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:10.375339 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc34af-f2d7-4d03-97eb-11da75c80bc3 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:10.375675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc34af-f2d7-4d03-97eb-11da75c80bc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:10.376843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc34af-f2d7-4d03-97eb-11da75c80bc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:10.386297 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a324e7f-9308-4535-8361-42ab4cc08ec9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:10.393368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a324e7f-9308-4535-8361-42ab4cc08ec9 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:10.393763 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc34af-f2d7-4d03-97eb-11da75c80bc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:10.396360 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc34af-f2d7-4d03-97eb-11da75c80bc3 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:10.414993 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 932/1820] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:09:10.429813 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 889/1821] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 588 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:09:10.466911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc34af-f2d7-4d03-97eb-11da75c80bc3 service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:10.468852 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5ddc34af-f2d7-4d03-97eb-11da75c80bc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:10.487818 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc34af-f2d7-4d03-97eb-11da75c80bc3 service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:10.492971 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12948cc-ed36-4b34-a0ec-c2075486078d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:10.493623 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12948cc-ed36-4b34-a0ec-c2075486078d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:10.493904 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12948cc-ed36-4b34-a0ec-c2075486078d admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:10.494178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12948cc-ed36-4b34-a0ec-c2075486078d admin admin] PATH_INFO: `/v3/users/266689c4a4864e90a3a719c52712aacf` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:10.495528 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12948cc-ed36-4b34-a0ec-c2075486078d admin admin] RBAC: Authorizing `identity:delete_user(user_id=266689c4a4864e90a3a719c52712aacf)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:10.497528 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12948cc-ed36-4b34-a0ec-c2075486078d admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:10.630155 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-c12948cc-ed36-4b34-a0ec-c2075486078d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '266689c4a4864e90a3a719c52712aacf'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:10.644107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-c12948cc-ed36-4b34-a0ec-c2075486078d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '266689c4a4864e90a3a719c52712aacf'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:10.662415 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-c12948cc-ed36-4b34-a0ec-c2075486078d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '266689c4a4864e90a3a719c52712aacf'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:10.663542 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-c12948cc-ed36-4b34-a0ec-c2075486078d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '266689c4a4864e90a3a719c52712aacf'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:10.664316 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-c12948cc-ed36-4b34-a0ec-c2075486078d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '266689c4a4864e90a3a719c52712aacf'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:10.675827 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50afe30-7340-4a74-9fba-1bbf3c8c51da None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:10.675827 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50afe30-7340-4a74-9fba-1bbf3c8c51da None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:10.675827 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50afe30-7340-4a74-9fba-1bbf3c8c51da None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:10.677049 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 890/1822] 10.222.0.36 () {54 vars in 921 bytes} [Fri Sep 2 09:09:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:09:10.688027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:10.761147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 933/1823] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:09:10 2022] DELETE /identity/v3/users/266689c4a4864e90a3a719c52712aacf => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:10.766798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e86707-b6cc-46f3-8eae-3455eafd8708 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:10.768956 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e86707-b6cc-46f3-8eae-3455eafd8708 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:10.769339 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e86707-b6cc-46f3-8eae-3455eafd8708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:10.807103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-c2e86707-b6cc-46f3-8eae-3455eafd8708 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:10.808582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c2e86707-b6cc-46f3-8eae-3455eafd8708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:10.843274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:10.877410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b5e9f-636d-4c94-be65-b3d2b8f1ea34 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:10.877888 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b5e9f-636d-4c94-be65-b3d2b8f1ea34 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:10.878285 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b5e9f-636d-4c94-be65-b3d2b8f1ea34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:10.914597 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-2a2b5e9f-636d-4c94-be65-b3d2b8f1ea34 None None] MFA Rules not processed for user `cce502a83f5343558c2136c0193e7d85`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:10.916718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-2a2b5e9f-636d-4c94-be65-b3d2b8f1ea34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:10.922416 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 891/1824] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:11.002058 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e86707-b6cc-46f3-8eae-3455eafd8708 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:11.005224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c2e86707-b6cc-46f3-8eae-3455eafd8708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:11.017758 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 934/1825] 10.222.0.36 () {58 vars in 1018 bytes} [Fri Sep 2 09:09:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:11.110990 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e86707-b6cc-46f3-8eae-3455eafd8708 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:11.116247 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f948a6-25f3-4e45-81f7-be7e0daa53b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:11.116894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f948a6-25f3-4e45-81f7-be7e0daa53b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:11.117179 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f948a6-25f3-4e45-81f7-be7e0daa53b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:11.117458 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f948a6-25f3-4e45-81f7-be7e0daa53b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:11.130626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f948a6-25f3-4e45-81f7-be7e0daa53b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:11.132306 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f948a6-25f3-4e45-81f7-be7e0daa53b4 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:11.146897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 892/1826] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:09:11.236691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b5e9f-636d-4c94-be65-b3d2b8f1ea34 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:11.238188 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-2a2b5e9f-636d-4c94-be65-b3d2b8f1ea34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:11.263862 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b5e9f-636d-4c94-be65-b3d2b8f1ea34 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:11.272103 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a7126-5c4e-43bf-8326-c88f61b9840c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:11.273067 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a7126-5c4e-43bf-8326-c88f61b9840c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:11.273443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a7126-5c4e-43bf-8326-c88f61b9840c admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:11.273811 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a7126-5c4e-43bf-8326-c88f61b9840c admin admin] PATH_INFO: `/v3/projects/1b21a798cdfb45d68573741d9b241ec5` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:11.275206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a7126-5c4e-43bf-8326-c88f61b9840c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b21a798cdfb45d68573741d9b241ec5)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:11.277438 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a7126-5c4e-43bf-8326-c88f61b9840c admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:11.328550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d73a7126-5c4e-43bf-8326-c88f61b9840c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b21a798cdfb45d68573741d9b241ec5', 'request_id': 'req-d73a7126-5c4e-43bf-8326-c88f61b9840c'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:11.337263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d73a7126-5c4e-43bf-8326-c88f61b9840c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b21a798cdfb45d68573741d9b241ec5', 'request_id': 'req-d73a7126-5c4e-43bf-8326-c88f61b9840c'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:11.348640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d73a7126-5c4e-43bf-8326-c88f61b9840c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b21a798cdfb45d68573741d9b241ec5', 'request_id': 'req-d73a7126-5c4e-43bf-8326-c88f61b9840c'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:11.360952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d73a7126-5c4e-43bf-8326-c88f61b9840c admin admin] The token cache is being invalidate because project 1b21a798cdfb45d68573741d9b241ec5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:09:11.361249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d73a7126-5c4e-43bf-8326-c88f61b9840c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:11.361918 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d73a7126-5c4e-43bf-8326-c88f61b9840c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:11.363593 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 935/1827] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:09:11 2022] DELETE /identity/v3/projects/1b21a798cdfb45d68573741d9b241ec5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:11.367850 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f948a6-25f3-4e45-81f7-be7e0daa53b4 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:11.380692 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f948a6-25f3-4e45-81f7-be7e0daa53b4 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:11.457780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1106a36-51ce-466e-8f38-5c6965e85d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:11.458495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1106a36-51ce-466e-8f38-5c6965e85d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:11.458849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1106a36-51ce-466e-8f38-5c6965e85d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:11.459224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1106a36-51ce-466e-8f38-5c6965e85d8b admin admin] PATH_INFO: `/v3/users/69a1b92c967d4827b5ab372ac12f20bf` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:11.460718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1106a36-51ce-466e-8f38-5c6965e85d8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=69a1b92c967d4827b5ab372ac12f20bf)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:11.463281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1106a36-51ce-466e-8f38-5c6965e85d8b admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:11.532830 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f1106a36-51ce-466e-8f38-5c6965e85d8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69a1b92c967d4827b5ab372ac12f20bf'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:11.533806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f1106a36-51ce-466e-8f38-5c6965e85d8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69a1b92c967d4827b5ab372ac12f20bf'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:11.546147 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f1106a36-51ce-466e-8f38-5c6965e85d8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69a1b92c967d4827b5ab372ac12f20bf'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:11.548955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:11.557277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f1106a36-51ce-466e-8f38-5c6965e85d8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69a1b92c967d4827b5ab372ac12f20bf'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:11.558121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f1106a36-51ce-466e-8f38-5c6965e85d8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69a1b92c967d4827b5ab372ac12f20bf'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:11.572416 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 893/1828] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:09:11 2022] DELETE /identity/v3/users/69a1b92c967d4827b5ab372ac12f20bf => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:12.070784 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a7126-5c4e-43bf-8326-c88f61b9840c admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:12.073099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-d73a7126-5c4e-43bf-8326-c88f61b9840c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:12.090217 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a7126-5c4e-43bf-8326-c88f61b9840c admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:12.198514 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b9c025-702e-438a-a438-d0b45ed847c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:12.200014 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9c025-702e-438a-a438-d0b45ed847c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:12.200462 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9c025-702e-438a-a438-d0b45ed847c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:12.200865 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b9c025-702e-438a-a438-d0b45ed847c5 admin admin] PATH_INFO: `/v3/projects/a59890050589479696934e66c2fbc97a` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:12.203476 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9c025-702e-438a-a438-d0b45ed847c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a59890050589479696934e66c2fbc97a)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:12.207605 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b9c025-702e-438a-a438-d0b45ed847c5 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:12.209661 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:12.291063 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-62b9c025-702e-438a-a438-d0b45ed847c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a59890050589479696934e66c2fbc97a', 'request_id': 'req-62b9c025-702e-438a-a438-d0b45ed847c5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:12.301536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1106a36-51ce-466e-8f38-5c6965e85d8b admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:12.304086 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-f1106a36-51ce-466e-8f38-5c6965e85d8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:12.305217 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-62b9c025-702e-438a-a438-d0b45ed847c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a59890050589479696934e66c2fbc97a', 'request_id': 'req-62b9c025-702e-438a-a438-d0b45ed847c5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:12.313919 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-62b9c025-702e-438a-a438-d0b45ed847c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a59890050589479696934e66c2fbc97a', 'request_id': 'req-62b9c025-702e-438a-a438-d0b45ed847c5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:12.323528 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1106a36-51ce-466e-8f38-5c6965e85d8b admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:12.327599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-62b9c025-702e-438a-a438-d0b45ed847c5 admin admin] The token cache is being invalidate because project a59890050589479696934e66c2fbc97a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:09:12.327969 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-62b9c025-702e-438a-a438-d0b45ed847c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:12.328720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-62b9c025-702e-438a-a438-d0b45ed847c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:12.334463 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 936/1829] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:09:12 2022] DELETE /identity/v3/projects/a59890050589479696934e66c2fbc97a => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:12.335855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e80a0a-3ddc-43b0-b543-fa7a30a128f5 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:12.336208 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e80a0a-3ddc-43b0-b543-fa7a30a128f5 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:12.336637 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e80a0a-3ddc-43b0-b543-fa7a30a128f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:12.362238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-75e80a0a-3ddc-43b0-b543-fa7a30a128f5 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:12.364105 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-75e80a0a-3ddc-43b0-b543-fa7a30a128f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:12.430132 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a64a0-4978-4adc-963b-9d22c98674ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:12.431402 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a64a0-4978-4adc-963b-9d22c98674ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:12.431734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a64a0-4978-4adc-963b-9d22c98674ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:12.432104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a64a0-4978-4adc-963b-9d22c98674ae admin admin] PATH_INFO: `/v3/projects/b1f68ddd905e4704b2d0b8d3beb7b2ec` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:12.433745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a64a0-4978-4adc-963b-9d22c98674ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1f68ddd905e4704b2d0b8d3beb7b2ec)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:12.437292 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90a64a0-4978-4adc-963b-9d22c98674ae admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:12.492992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 937/1830] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:09:12.503851 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9f628-6590-418b-973e-5d7d4aa19def None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:12.504672 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9f628-6590-418b-973e-5d7d4aa19def None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:12.505172 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9f628-6590-418b-973e-5d7d4aa19def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:12.519182 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f90a64a0-4978-4adc-963b-9d22c98674ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1f68ddd905e4704b2d0b8d3beb7b2ec', 'request_id': 'req-f90a64a0-4978-4adc-963b-9d22c98674ae'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:12.531164 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f90a64a0-4978-4adc-963b-9d22c98674ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1f68ddd905e4704b2d0b8d3beb7b2ec', 'request_id': 'req-f90a64a0-4978-4adc-963b-9d22c98674ae'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:12.539999 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f90a64a0-4978-4adc-963b-9d22c98674ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1f68ddd905e4704b2d0b8d3beb7b2ec', 'request_id': 'req-f90a64a0-4978-4adc-963b-9d22c98674ae'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:12.540652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-7dc9f628-6590-418b-973e-5d7d4aa19def None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:12.541820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-7dc9f628-6590-418b-973e-5d7d4aa19def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:12.563030 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f90a64a0-4978-4adc-963b-9d22c98674ae admin admin] The token cache is being invalidate because project b1f68ddd905e4704b2d0b8d3beb7b2ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:09:12.563418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f90a64a0-4978-4adc-963b-9d22c98674ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:12.567454 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f90a64a0-4978-4adc-963b-9d22c98674ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:12.571137 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 894/1831] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:09:12 2022] DELETE /identity/v3/projects/b1f68ddd905e4704b2d0b8d3beb7b2ec => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:12.575578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6651c7-ada2-4b9e-8df6-7bc698788c89 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:12.576600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6651c7-ada2-4b9e-8df6-7bc698788c89 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:12.577018 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6651c7-ada2-4b9e-8df6-7bc698788c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:12.622997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-0c6651c7-ada2-4b9e-8df6-7bc698788c89 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:12.627166 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-0c6651c7-ada2-4b9e-8df6-7bc698788c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:12.667788 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 938/1832] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:09:12.669133 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9f628-6590-418b-973e-5d7d4aa19def None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:12.671809 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-7dc9f628-6590-418b-973e-5d7d4aa19def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:12.730963 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:12.771295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 895/1833] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:09:12.772185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6651c7-ada2-4b9e-8df6-7bc698788c89 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:12.785809 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6651c7-ada2-4b9e-8df6-7bc698788c89 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:12.791096 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096d6987-bfd1-4182-b85e-cb610b854213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:12.791770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d6987-bfd1-4182-b85e-cb610b854213 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:12.792053 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d6987-bfd1-4182-b85e-cb610b854213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:12.792385 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096d6987-bfd1-4182-b85e-cb610b854213 admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:12.793040 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d6987-bfd1-4182-b85e-cb610b854213 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:09:12.793339 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d6987-bfd1-4182-b85e-cb610b854213 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:12.794987 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096d6987-bfd1-4182-b85e-cb610b854213 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:12.804002 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9f628-6590-418b-973e-5d7d4aa19def None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:12.812123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3adbe52-3c2e-4386-88f1-d217f5c88a02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:12.812814 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3adbe52-3c2e-4386-88f1-d217f5c88a02 service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:12.813165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3adbe52-3c2e-4386-88f1-d217f5c88a02 service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:12.813558 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3adbe52-3c2e-4386-88f1-d217f5c88a02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:12.816530 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-b3adbe52-3c2e-4386-88f1-d217f5c88a02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:12.826959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 896/1834] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:09:12.828717 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e505598-fc8b-48d4-8c4e-9a306a0e377e None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:12.828717 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e505598-fc8b-48d4-8c4e-9a306a0e377e None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:12.828863 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e505598-fc8b-48d4-8c4e-9a306a0e377e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:12.841555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:12.855967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3adbe52-3c2e-4386-88f1-d217f5c88a02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:12.861523 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-9e505598-fc8b-48d4-8c4e-9a306a0e377e None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:12.862230 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3adbe52-3c2e-4386-88f1-d217f5c88a02 service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:12.865335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-9e505598-fc8b-48d4-8c4e-9a306a0e377e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:12.876033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 897/1835] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:12.877087 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e505598-fc8b-48d4-8c4e-9a306a0e377e None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:12.887559 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e505598-fc8b-48d4-8c4e-9a306a0e377e None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:12.890819 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 939/1836] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:09:12.892886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3adbe52-3c2e-4386-88f1-d217f5c88a02 service nova] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:12.909028 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91db17-e121-40b8-a7f6-01357719ecfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:12.909692 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91db17-e121-40b8-a7f6-01357719ecfa admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:12.910015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91db17-e121-40b8-a7f6-01357719ecfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:12.910314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91db17-e121-40b8-a7f6-01357719ecfa admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:12.911269 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91db17-e121-40b8-a7f6-01357719ecfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:12.913464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91db17-e121-40b8-a7f6-01357719ecfa admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:12.917104 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3adbe52-3c2e-4386-88f1-d217f5c88a02 service nova] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:12.932847 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aeb77b-92e7-46a0-b997-a8c789279c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:12.933655 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aeb77b-92e7-46a0-b997-a8c789279c14 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:12.934099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aeb77b-92e7-46a0-b997-a8c789279c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:12.934373 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aeb77b-92e7-46a0-b997-a8c789279c14 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:12.935017 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aeb77b-92e7-46a0-b997-a8c789279c14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:09:12.935351 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aeb77b-92e7-46a0-b997-a8c789279c14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:12.937336 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aeb77b-92e7-46a0-b997-a8c789279c14 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:12.977831 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 940/1837] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:09:12.984390 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aeb77b-92e7-46a0-b997-a8c789279c14 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:13.013708 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 898/1838] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:09:12 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:09:13.015008 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aeb77b-92e7-46a0-b997-a8c789279c14 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:13.019006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91db17-e121-40b8-a7f6-01357719ecfa admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:13.130304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015dbb1-d69d-472e-b5ee-379a9e3635b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:13.131246 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015dbb1-d69d-472e-b5ee-379a9e3635b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:13.131790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015dbb1-d69d-472e-b5ee-379a9e3635b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:13.132442 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015dbb1-d69d-472e-b5ee-379a9e3635b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:13.133391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015dbb1-d69d-472e-b5ee-379a9e3635b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:13.136385 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015dbb1-d69d-472e-b5ee-379a9e3635b0 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:13.154124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91db17-e121-40b8-a7f6-01357719ecfa admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:13.171224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decc534-9ac0-4b2e-9d5c-56addb4859ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:13.171224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decc534-9ac0-4b2e-9d5c-56addb4859ac admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:13.171224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decc534-9ac0-4b2e-9d5c-56addb4859ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:13.171224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decc534-9ac0-4b2e-9d5c-56addb4859ac admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:13.171734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4decc534-9ac0-4b2e-9d5c-56addb4859ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:13.171734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4decc534-9ac0-4b2e-9d5c-56addb4859ac admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:13.199733 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 941/1839] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:09:12 2022] POST /identity/v3/projects => generated 373 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:09:13.200936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015dbb1-d69d-472e-b5ee-379a9e3635b0 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:13.207470 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 899/1840] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:09:13 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:09:13.211179 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decc534-9ac0-4b2e-9d5c-56addb4859ac admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:13.224855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decc534-9ac0-4b2e-9d5c-56addb4859ac admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:13.316865 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcb60c-fc32-4be9-a97d-a8b4dd7adc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:13.317596 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcb60c-fc32-4be9-a97d-a8b4dd7adc74 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:13.317978 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcb60c-fc32-4be9-a97d-a8b4dd7adc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:13.318492 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fcb60c-fc32-4be9-a97d-a8b4dd7adc74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:13.319373 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcb60c-fc32-4be9-a97d-a8b4dd7adc74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:13.321253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fcb60c-fc32-4be9-a97d-a8b4dd7adc74 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:13.350398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 900/1841] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:09:13.355249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015dbb1-d69d-472e-b5ee-379a9e3635b0 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:13.356138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcb60c-fc32-4be9-a97d-a8b4dd7adc74 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:13.371245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fcb60c-fc32-4be9-a97d-a8b4dd7adc74 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:13.372726 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66949c7-ea85-4a87-866a-d0a3fb0b5696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:13.373441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66949c7-ea85-4a87-866a-d0a3fb0b5696 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:13.373731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66949c7-ea85-4a87-866a-d0a3fb0b5696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:13.374002 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66949c7-ea85-4a87-866a-d0a3fb0b5696 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:13.374778 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66949c7-ea85-4a87-866a-d0a3fb0b5696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:13.377832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97845c1d-a12c-472f-b288-05a52981dc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:13.378783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66949c7-ea85-4a87-866a-d0a3fb0b5696 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:13.378922 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97845c1d-a12c-472f-b288-05a52981dc3a admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:13.379297 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97845c1d-a12c-472f-b288-05a52981dc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:13.379671 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97845c1d-a12c-472f-b288-05a52981dc3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:13.380603 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97845c1d-a12c-472f-b288-05a52981dc3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:13.382485 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97845c1d-a12c-472f-b288-05a52981dc3a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:13.404711 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 901/1842] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:09:13.405774 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97845c1d-a12c-472f-b288-05a52981dc3a admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:13.412285 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 942/1843] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:09:13 2022] POST /identity/v3/users => generated 287 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:09:13.413483 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66949c7-ea85-4a87-866a-d0a3fb0b5696 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:13.417391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97845c1d-a12c-472f-b288-05a52981dc3a admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:13.423416 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6715dcb6-f5cd-452e-83e1-7c5ce02ef92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:13.425112 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715dcb6-f5cd-452e-83e1-7c5ce02ef92a admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:13.425585 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715dcb6-f5cd-452e-83e1-7c5ce02ef92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:13.425959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6715dcb6-f5cd-452e-83e1-7c5ce02ef92a admin admin] PATH_INFO: `/v3/projects/6ab762666fba484d9e09baeb2a928769/users/17c8716fda5f4cadb81ee05bd99a77f3/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:13.427344 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66949c7-ea85-4a87-866a-d0a3fb0b5696 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:13.432647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cacf273-b8e8-412f-bc15-ad96f4d45ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:13.433428 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacf273-b8e8-412f-bc15-ad96f4d45ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:13.433712 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacf273-b8e8-412f-bc15-ad96f4d45ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:13.433985 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacf273-b8e8-412f-bc15-ad96f4d45ba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:13.434635 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cacf273-b8e8-412f-bc15-ad96f4d45ba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:13.436462 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cacf273-b8e8-412f-bc15-ad96f4d45ba5 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:13.447708 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715dcb6-f5cd-452e-83e1-7c5ce02ef92a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ab762666fba484d9e09baeb2a928769, user_id=17c8716fda5f4cadb81ee05bd99a77f3, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:13.451399 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6715dcb6-f5cd-452e-83e1-7c5ce02ef92a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:13.453828 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 943/1844] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:09:13.454861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cacf273-b8e8-412f-bc15-ad96f4d45ba5 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:13.478988 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cacf273-b8e8-412f-bc15-ad96f4d45ba5 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:13.480777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 902/1845] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:09:13 2022] PUT /identity/v3/projects/6ab762666fba484d9e09baeb2a928769/users/17c8716fda5f4cadb81ee05bd99a77f3/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:13.483255 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6faa026-425a-47b2-a49d-f0325302004f None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:13.483656 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6faa026-425a-47b2-a49d-f0325302004f None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:13.484030 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6faa026-425a-47b2-a49d-f0325302004f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:13.484676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:13.524091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-e6faa026-425a-47b2-a49d-f0325302004f None None] MFA Rules not processed for user `17c8716fda5f4cadb81ee05bd99a77f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:13.525746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-e6faa026-425a-47b2-a49d-f0325302004f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:13.538678 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:13.567246 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407194c-0529-4fba-97e7-70c8b26dbf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:13.567886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407194c-0529-4fba-97e7-70c8b26dbf8b admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:13.568232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407194c-0529-4fba-97e7-70c8b26dbf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:13.568576 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407194c-0529-4fba-97e7-70c8b26dbf8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:13.569272 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407194c-0529-4fba-97e7-70c8b26dbf8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:13.570933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3407194c-0529-4fba-97e7-70c8b26dbf8b admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:13.581685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 944/1846] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:09:13.585952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407194c-0529-4fba-97e7-70c8b26dbf8b admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:13.595943 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407194c-0529-4fba-97e7-70c8b26dbf8b admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:13.600670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa3291a-9e28-45c5-ac19-e4973e4efd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:13.601311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3291a-9e28-45c5-ac19-e4973e4efd29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:13.601591 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 903/1847] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:13.601723 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3291a-9e28-45c5-ac19-e4973e4efd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:13.601892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa3291a-9e28-45c5-ac19-e4973e4efd29 admin admin] PATH_INFO: `/v3/projects/606a7f5c127742ca809573202fb8bdf8/users/0b41ae57d4964c02970bb1acd6569006/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:13.606019 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6faa026-425a-47b2-a49d-f0325302004f None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:13.616798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6faa026-425a-47b2-a49d-f0325302004f None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:13.617203 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3291a-9e28-45c5-ac19-e4973e4efd29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=606a7f5c127742ca809573202fb8bdf8, user_id=0b41ae57d4964c02970bb1acd6569006, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:13.620036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa3291a-9e28-45c5-ac19-e4973e4efd29 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:13.621415 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4360d46-0696-4282-ac85-64e36ea127a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:13.622122 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4360d46-0696-4282-ac85-64e36ea127a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:13.622292 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4360d46-0696-4282-ac85-64e36ea127a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:13.622561 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4360d46-0696-4282-ac85-64e36ea127a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:13.623345 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4360d46-0696-4282-ac85-64e36ea127a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:13.625220 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4360d46-0696-4282-ac85-64e36ea127a4 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:13.633050 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 945/1848] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:09:13 2022] PUT /identity/v3/projects/606a7f5c127742ca809573202fb8bdf8/users/0b41ae57d4964c02970bb1acd6569006/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:09:13.644753 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff7e2d-5e30-4671-a70f-db4c162e709e None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:13.645036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff7e2d-5e30-4671-a70f-db4c162e709e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:13.645308 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff7e2d-5e30-4671-a70f-db4c162e709e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:13.659581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 904/1849] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:09:13 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:09:13.663033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4360d46-0696-4282-ac85-64e36ea127a4 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:13.683316 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-01ff7e2d-5e30-4671-a70f-db4c162e709e None None] MFA Rules not processed for user `0b41ae57d4964c02970bb1acd6569006`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:13.685183 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-01ff7e2d-5e30-4671-a70f-db4c162e709e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:13.742042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4360d46-0696-4282-ac85-64e36ea127a4 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:13.747381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14431691-afba-41e1-aeee-10ebba121696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:13.748288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14431691-afba-41e1-aeee-10ebba121696 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:13.748571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14431691-afba-41e1-aeee-10ebba121696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:13.748846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14431691-afba-41e1-aeee-10ebba121696 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:13.749485 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14431691-afba-41e1-aeee-10ebba121696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:13.752068 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14431691-afba-41e1-aeee-10ebba121696 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:13.762266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 946/1850] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:09:13.766830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff7e2d-5e30-4671-a70f-db4c162e709e None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:13.775870 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 905/1851] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:09:13 2022] POST /identity/v3/users => generated 298 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:09:13.779923 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff7e2d-5e30-4671-a70f-db4c162e709e None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:13.780152 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14431691-afba-41e1-aeee-10ebba121696 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:13.787715 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e41c9-b750-467a-ba2e-0faf07cbae7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:13.788395 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e41c9-b750-467a-ba2e-0faf07cbae7c admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:13.788700 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e41c9-b750-467a-ba2e-0faf07cbae7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:13.788993 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e41c9-b750-467a-ba2e-0faf07cbae7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:13.789856 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e41c9-b750-467a-ba2e-0faf07cbae7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:13.792019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e41c9-b750-467a-ba2e-0faf07cbae7c admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:13.792923 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14431691-afba-41e1-aeee-10ebba121696 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:13.798920 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba338aba-7a49-42eb-b188-eb55f5873b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:13.799566 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba338aba-7a49-42eb-b188-eb55f5873b0c admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:13.799850 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba338aba-7a49-42eb-b188-eb55f5873b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:13.800122 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba338aba-7a49-42eb-b188-eb55f5873b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:13.800765 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba338aba-7a49-42eb-b188-eb55f5873b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:13.802368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba338aba-7a49-42eb-b188-eb55f5873b0c admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:13.814516 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 906/1852] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:09:13.818844 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba338aba-7a49-42eb-b188-eb55f5873b0c admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:13.822529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 947/1853] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:09:13 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:09:13.826180 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e41c9-b750-467a-ba2e-0faf07cbae7c admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:13.829334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba338aba-7a49-42eb-b188-eb55f5873b0c admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:13.875052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:13.891317 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e41c9-b750-467a-ba2e-0faf07cbae7c admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:13.893666 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8d32b-49c1-4cc4-88bc-9939fe332417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:13.894552 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8d32b-49c1-4cc4-88bc-9939fe332417 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:13.894934 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8d32b-49c1-4cc4-88bc-9939fe332417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:13.895347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8d32b-49c1-4cc4-88bc-9939fe332417 admin admin] PATH_INFO: `/v3/projects/8f4497b720094374b4c6cf083c29268c/users/c614be28be434e1cbf4ee8b175db2881/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:13.896139 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61153e4-3d33-44a2-9040-3c50aebf3684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:13.896745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61153e4-3d33-44a2-9040-3c50aebf3684 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:13.897026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61153e4-3d33-44a2-9040-3c50aebf3684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:13.897296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61153e4-3d33-44a2-9040-3c50aebf3684 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:13.897960 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61153e4-3d33-44a2-9040-3c50aebf3684 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:13.899915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61153e4-3d33-44a2-9040-3c50aebf3684 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:13.912223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8d32b-49c1-4cc4-88bc-9939fe332417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f4497b720094374b4c6cf083c29268c, user_id=c614be28be434e1cbf4ee8b175db2881, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:13.915439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8d32b-49c1-4cc4-88bc-9939fe332417 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:13.917867 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:13.923322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 948/1854] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:09:13 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:09:13.927200 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61153e4-3d33-44a2-9040-3c50aebf3684 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:13.930801 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 907/1855] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:09:13 2022] PUT /identity/v3/projects/8f4497b720094374b4c6cf083c29268c/users/c614be28be434e1cbf4ee8b175db2881/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:09:13.934724 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8d32b-49c1-4cc4-88bc-9939fe332417 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:13.937522 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61153e4-3d33-44a2-9040-3c50aebf3684 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:13.994769 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53e9ac-3462-421d-b26d-14da2bd848a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:13.995411 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53e9ac-3462-421d-b26d-14da2bd848a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:13.995688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53e9ac-3462-421d-b26d-14da2bd848a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:13.995964 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53e9ac-3462-421d-b26d-14da2bd848a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:13.996647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f53e9ac-3462-421d-b26d-14da2bd848a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:13.998074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8d32b-49c1-4cc4-88bc-9939fe332417 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:13.998250 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f53e9ac-3462-421d-b26d-14da2bd848a9 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:14.002825 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf7f78-586f-4a13-93c3-d2f15f056cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:14.003513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf7f78-586f-4a13-93c3-d2f15f056cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:14.003727 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf7f78-586f-4a13-93c3-d2f15f056cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:14.004002 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf7f78-586f-4a13-93c3-d2f15f056cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:14.004643 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf7f78-586f-4a13-93c3-d2f15f056cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:14.006244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf7f78-586f-4a13-93c3-d2f15f056cf8 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:14.008006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 949/1856] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:09:14.012117 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53e9ac-3462-421d-b26d-14da2bd848a9 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:14.019508 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 908/1857] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:09:14.023287 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf7f78-586f-4a13-93c3-d2f15f056cf8 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:14.024261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53e9ac-3462-421d-b26d-14da2bd848a9 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:14.029092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807ab12-4e61-4c67-bd2f-4d712e96f884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:14.029884 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807ab12-4e61-4c67-bd2f-4d712e96f884 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:14.030194 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807ab12-4e61-4c67-bd2f-4d712e96f884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:14.030470 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807ab12-4e61-4c67-bd2f-4d712e96f884 admin admin] PATH_INFO: `/v3/projects/4f26863d92414fa2b7522dabe0d1bdb3/users/06188fe0f2f44142a3d3f1a3bf859faf/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:14.033298 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf7f78-586f-4a13-93c3-d2f15f056cf8 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:14.037922 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace3ff3e-8f8f-4976-890b-6225232dfd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:14.038546 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3ff3e-8f8f-4976-890b-6225232dfd9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:14.038825 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3ff3e-8f8f-4976-890b-6225232dfd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:14.039127 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace3ff3e-8f8f-4976-890b-6225232dfd9e admin admin] PATH_INFO: `/v3/projects/8f4497b720094374b4c6cf083c29268c/users/c614be28be434e1cbf4ee8b175db2881/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:14.041031 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace3ff3e-8f8f-4976-890b-6225232dfd9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f4497b720094374b4c6cf083c29268c, user_id=c614be28be434e1cbf4ee8b175db2881, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:14.043690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace3ff3e-8f8f-4976-890b-6225232dfd9e admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:14.046150 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807ab12-4e61-4c67-bd2f-4d712e96f884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f26863d92414fa2b7522dabe0d1bdb3, user_id=06188fe0f2f44142a3d3f1a3bf859faf, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:14.049675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807ab12-4e61-4c67-bd2f-4d712e96f884 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:14.054620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 909/1858] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:09:14 2022] PUT /identity/v3/projects/8f4497b720094374b4c6cf083c29268c/users/c614be28be434e1cbf4ee8b175db2881/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:09:14.063904 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 950/1859] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:09:14 2022] PUT /identity/v3/projects/4f26863d92414fa2b7522dabe0d1bdb3/users/06188fe0f2f44142a3d3f1a3bf859faf/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:09:14.065394 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcb558-654a-456c-8dc8-bda17eb7228c None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:14.065685 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcb558-654a-456c-8dc8-bda17eb7228c None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:14.065958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcb558-654a-456c-8dc8-bda17eb7228c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:14.068737 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807ab12-4e61-4c67-bd2f-4d712e96f884 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:14.095171 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-12fcb558-654a-456c-8dc8-bda17eb7228c None None] MFA Rules not processed for user `c614be28be434e1cbf4ee8b175db2881`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:14.096348 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-12fcb558-654a-456c-8dc8-bda17eb7228c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:14.135122 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807ab12-4e61-4c67-bd2f-4d712e96f884 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:14.139771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857fc52-4809-43c7-b1e5-a8723457e2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:14.140375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857fc52-4809-43c7-b1e5-a8723457e2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:14.140652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857fc52-4809-43c7-b1e5-a8723457e2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:14.140926 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857fc52-4809-43c7-b1e5-a8723457e2e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:14.141556 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2857fc52-4809-43c7-b1e5-a8723457e2e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:14.143176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2857fc52-4809-43c7-b1e5-a8723457e2e0 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:14.152964 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 951/1860] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:09:14.156910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857fc52-4809-43c7-b1e5-a8723457e2e0 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:14.163770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 910/1861] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:09:14.167892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857fc52-4809-43c7-b1e5-a8723457e2e0 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:14.172531 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbb5fde-464a-4068-8937-8a1c38733239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:14.173749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb5fde-464a-4068-8937-8a1c38733239 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:14.174031 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb5fde-464a-4068-8937-8a1c38733239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:14.174305 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb5fde-464a-4068-8937-8a1c38733239 admin admin] PATH_INFO: `/v3/projects/4f26863d92414fa2b7522dabe0d1bdb3/users/06188fe0f2f44142a3d3f1a3bf859faf/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:14.175681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fcb558-654a-456c-8dc8-bda17eb7228c None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:14.176610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb5fde-464a-4068-8937-8a1c38733239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f26863d92414fa2b7522dabe0d1bdb3, user_id=06188fe0f2f44142a3d3f1a3bf859faf, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:14.177207 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-12fcb558-654a-456c-8dc8-bda17eb7228c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:14.179215 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbb5fde-464a-4068-8937-8a1c38733239 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:14.191153 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 952/1862] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:09:14 2022] PUT /identity/v3/projects/4f26863d92414fa2b7522dabe0d1bdb3/users/06188fe0f2f44142a3d3f1a3bf859faf/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:09:14.201961 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009614d0-0d55-498e-8ab1-b799dd3b6df6 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:14.202250 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009614d0-0d55-498e-8ab1-b799dd3b6df6 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:14.202524 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009614d0-0d55-498e-8ab1-b799dd3b6df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:14.232076 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-009614d0-0d55-498e-8ab1-b799dd3b6df6 None None] MFA Rules not processed for user `06188fe0f2f44142a3d3f1a3bf859faf`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:14.233298 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-009614d0-0d55-498e-8ab1-b799dd3b6df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:14.242726 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fcb558-654a-456c-8dc8-bda17eb7228c None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:14.252652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:14.260148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:14.307952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 953/1863] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:14.320073 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009614d0-0d55-498e-8ab1-b799dd3b6df6 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:14.321972 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba04ef-492d-4420-b987-686578308b40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:14.322467 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-009614d0-0d55-498e-8ab1-b799dd3b6df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:14.322615 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba04ef-492d-4420-b987-686578308b40 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:14.322895 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba04ef-492d-4420-b987-686578308b40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:14.323199 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba04ef-492d-4420-b987-686578308b40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:14.337089 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009614d0-0d55-498e-8ab1-b799dd3b6df6 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:14.341082 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d9b4d-9be9-428e-bde1-2a27b7b8f85e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:14.341881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d9b4d-9be9-428e-bde1-2a27b7b8f85e service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:14.342267 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d9b4d-9be9-428e-bde1-2a27b7b8f85e service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:14.342639 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d9b4d-9be9-428e-bde1-2a27b7b8f85e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:14.403283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba04ef-492d-4420-b987-686578308b40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:14.405313 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba04ef-492d-4420-b987-686578308b40 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:14.415740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d9b4d-9be9-428e-bde1-2a27b7b8f85e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:14.417195 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 911/1864] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:14 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:09:14.417883 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d9b4d-9be9-428e-bde1-2a27b7b8f85e service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:14.428360 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 954/1865] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:09:14.543046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba04ef-492d-4420-b987-686578308b40 service cinder] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:14.552441 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba04ef-492d-4420-b987-686578308b40 service cinder] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:14.556119 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce4f0b-4520-486b-9d5e-06bdc17c3e9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:14.556738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce4f0b-4520-486b-9d5e-06bdc17c3e9b service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:14.557014 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce4f0b-4520-486b-9d5e-06bdc17c3e9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:14.557289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce4f0b-4520-486b-9d5e-06bdc17c3e9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:14.636994 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce4f0b-4520-486b-9d5e-06bdc17c3e9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:14.638817 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce4f0b-4520-486b-9d5e-06bdc17c3e9b service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:14.651975 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 912/1866] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:09:14.977423 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d9b4d-9be9-428e-bde1-2a27b7b8f85e service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:15.040548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d9b4d-9be9-428e-bde1-2a27b7b8f85e service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:15.044987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724bb7b2-fb55-4dc1-8fe0-1a24299afa30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:15.045614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bb7b2-fb55-4dc1-8fe0-1a24299afa30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:15.045894 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bb7b2-fb55-4dc1-8fe0-1a24299afa30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:15.046163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bb7b2-fb55-4dc1-8fe0-1a24299afa30 admin admin] PATH_INFO: `/v3/users/17c8716fda5f4cadb81ee05bd99a77f3` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:15.047285 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724bb7b2-fb55-4dc1-8fe0-1a24299afa30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17c8716fda5f4cadb81ee05bd99a77f3)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:15.049214 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724bb7b2-fb55-4dc1-8fe0-1a24299afa30 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:15.094443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-724bb7b2-fb55-4dc1-8fe0-1a24299afa30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17c8716fda5f4cadb81ee05bd99a77f3'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:15.101350 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-724bb7b2-fb55-4dc1-8fe0-1a24299afa30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17c8716fda5f4cadb81ee05bd99a77f3'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:15.102986 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:15.108985 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-724bb7b2-fb55-4dc1-8fe0-1a24299afa30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17c8716fda5f4cadb81ee05bd99a77f3'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:15.110047 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-724bb7b2-fb55-4dc1-8fe0-1a24299afa30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17c8716fda5f4cadb81ee05bd99a77f3'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:15.111033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-724bb7b2-fb55-4dc1-8fe0-1a24299afa30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17c8716fda5f4cadb81ee05bd99a77f3'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:15.128268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 955/1867] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:09:14 2022] DELETE /identity/v3/users/17c8716fda5f4cadb81ee05bd99a77f3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:09:15.132943 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164a25b-b0f5-4a75-a10c-b844a68bc9b2 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:15.133171 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164a25b-b0f5-4a75-a10c-b844a68bc9b2 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:15.133447 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164a25b-b0f5-4a75-a10c-b844a68bc9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:15.151926 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-0164a25b-b0f5-4a75-a10c-b844a68bc9b2 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:15.153472 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-0164a25b-b0f5-4a75-a10c-b844a68bc9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:15.216137 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 913/1868] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:15.234193 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724bb7b2-fb55-4dc1-8fe0-1a24299afa30 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:15.236262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-724bb7b2-fb55-4dc1-8fe0-1a24299afa30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:15.293557 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724bb7b2-fb55-4dc1-8fe0-1a24299afa30 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:15.297237 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511445c0-75b3-4610-b1f1-872f7bc84b4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:15.298447 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511445c0-75b3-4610-b1f1-872f7bc84b4f service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:15.298799 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511445c0-75b3-4610-b1f1-872f7bc84b4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:15.299094 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511445c0-75b3-4610-b1f1-872f7bc84b4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:15.309183 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511445c0-75b3-4610-b1f1-872f7bc84b4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:15.310650 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511445c0-75b3-4610-b1f1-872f7bc84b4f service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:15.320973 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 956/1869] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:09:16.752625 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164a25b-b0f5-4a75-a10c-b844a68bc9b2 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:16.754185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-0164a25b-b0f5-4a75-a10c-b844a68bc9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:16.767418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164a25b-b0f5-4a75-a10c-b844a68bc9b2 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:16.791685 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f639c0c-c4d6-40b5-988e-673a0609bcad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:16.792240 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f639c0c-c4d6-40b5-988e-673a0609bcad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:16.792762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f639c0c-c4d6-40b5-988e-673a0609bcad admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:16.792762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f639c0c-c4d6-40b5-988e-673a0609bcad admin admin] PATH_INFO: `/v3/projects/6ab762666fba484d9e09baeb2a928769` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:16.794560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f639c0c-c4d6-40b5-988e-673a0609bcad admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ab762666fba484d9e09baeb2a928769)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:16.796512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f639c0c-c4d6-40b5-988e-673a0609bcad admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:16.798313 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:16.880911 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7f639c0c-c4d6-40b5-988e-673a0609bcad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ab762666fba484d9e09baeb2a928769', 'request_id': 'req-7f639c0c-c4d6-40b5-988e-673a0609bcad'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:16.887702 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7f639c0c-c4d6-40b5-988e-673a0609bcad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ab762666fba484d9e09baeb2a928769', 'request_id': 'req-7f639c0c-c4d6-40b5-988e-673a0609bcad'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:16.893871 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7f639c0c-c4d6-40b5-988e-673a0609bcad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ab762666fba484d9e09baeb2a928769', 'request_id': 'req-7f639c0c-c4d6-40b5-988e-673a0609bcad'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:16.917557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7f639c0c-c4d6-40b5-988e-673a0609bcad admin admin] The token cache is being invalidate because project 6ab762666fba484d9e09baeb2a928769 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:09:16.917923 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7f639c0c-c4d6-40b5-988e-673a0609bcad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:16.918577 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7f639c0c-c4d6-40b5-988e-673a0609bcad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:16.920172 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 914/1870] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:09:16 2022] DELETE /identity/v3/projects/6ab762666fba484d9e09baeb2a928769 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:16.924167 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511445c0-75b3-4610-b1f1-872f7bc84b4f service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:16.934662 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511445c0-75b3-4610-b1f1-872f7bc84b4f service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:16.994625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d61311-f741-4000-9e9a-25e71eed2b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:16.995381 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d61311-f741-4000-9e9a-25e71eed2b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:16.995657 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d61311-f741-4000-9e9a-25e71eed2b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:16.995924 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d61311-f741-4000-9e9a-25e71eed2b33 admin admin] PATH_INFO: `/v3/users/c614be28be434e1cbf4ee8b175db2881` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:16.997024 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d61311-f741-4000-9e9a-25e71eed2b33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c614be28be434e1cbf4ee8b175db2881)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:16.998971 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d61311-f741-4000-9e9a-25e71eed2b33 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:17.044812 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-55d61311-f741-4000-9e9a-25e71eed2b33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c614be28be434e1cbf4ee8b175db2881'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:17.051772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-55d61311-f741-4000-9e9a-25e71eed2b33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c614be28be434e1cbf4ee8b175db2881'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:17.058979 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-55d61311-f741-4000-9e9a-25e71eed2b33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c614be28be434e1cbf4ee8b175db2881'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:17.059613 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-55d61311-f741-4000-9e9a-25e71eed2b33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c614be28be434e1cbf4ee8b175db2881'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:17.060212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-55d61311-f741-4000-9e9a-25e71eed2b33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c614be28be434e1cbf4ee8b175db2881'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:17.071466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:17.080966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 957/1871] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:09:16 2022] DELETE /identity/v3/users/c614be28be434e1cbf4ee8b175db2881 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:17.270370 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f639c0c-c4d6-40b5-988e-673a0609bcad admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:17.273165 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7f639c0c-c4d6-40b5-988e-673a0609bcad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:17.288987 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f639c0c-c4d6-40b5-988e-673a0609bcad admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:17.431512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa1496e-8af7-4154-85bc-00736a37749c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:17.432305 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1496e-8af7-4154-85bc-00736a37749c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:17.432651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1496e-8af7-4154-85bc-00736a37749c admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:17.433073 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa1496e-8af7-4154-85bc-00736a37749c admin admin] PATH_INFO: `/v3/users/0b41ae57d4964c02970bb1acd6569006` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:17.434634 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1496e-8af7-4154-85bc-00736a37749c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b41ae57d4964c02970bb1acd6569006)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:17.437884 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa1496e-8af7-4154-85bc-00736a37749c admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:17.516444 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d61311-f741-4000-9e9a-25e71eed2b33 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:17.518996 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-55d61311-f741-4000-9e9a-25e71eed2b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:17.568134 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d61311-f741-4000-9e9a-25e71eed2b33 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:17.579237 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-efa1496e-8af7-4154-85bc-00736a37749c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b41ae57d4964c02970bb1acd6569006'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:17.580727 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-efa1496e-8af7-4154-85bc-00736a37749c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b41ae57d4964c02970bb1acd6569006'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:17.582340 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbbe0b5-8e14-41b3-bf03-b07638248ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:17.583146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbe0b5-8e14-41b3-bf03-b07638248ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:17.583465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbe0b5-8e14-41b3-bf03-b07638248ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:17.583798 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbbe0b5-8e14-41b3-bf03-b07638248ad3 admin admin] PATH_INFO: `/v3/projects/8f4497b720094374b4c6cf083c29268c` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:17.592107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbbe0b5-8e14-41b3-bf03-b07638248ad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f4497b720094374b4c6cf083c29268c)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:17.594493 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbbe0b5-8e14-41b3-bf03-b07638248ad3 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:17.613824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-efa1496e-8af7-4154-85bc-00736a37749c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b41ae57d4964c02970bb1acd6569006'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:17.630522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-efa1496e-8af7-4154-85bc-00736a37749c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b41ae57d4964c02970bb1acd6569006'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:17.632663 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-efa1496e-8af7-4154-85bc-00736a37749c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b41ae57d4964c02970bb1acd6569006'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:17.656970 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 915/1872] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:09:17 2022] DELETE /identity/v3/users/0b41ae57d4964c02970bb1acd6569006 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:17.658691 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011c22a-c6e1-4a94-8916-c38bf63c4683 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:17.659555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011c22a-c6e1-4a94-8916-c38bf63c4683 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:17.659998 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011c22a-c6e1-4a94-8916-c38bf63c4683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:17.723602 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-c011c22a-c6e1-4a94-8916-c38bf63c4683 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:17.725512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c011c22a-c6e1-4a94-8916-c38bf63c4683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:17.727901 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-2dbbe0b5-8e14-41b3-bf03-b07638248ad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f4497b720094374b4c6cf083c29268c', 'request_id': 'req-2dbbe0b5-8e14-41b3-bf03-b07638248ad3'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:17.739099 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:17.741633 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-2dbbe0b5-8e14-41b3-bf03-b07638248ad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f4497b720094374b4c6cf083c29268c', 'request_id': 'req-2dbbe0b5-8e14-41b3-bf03-b07638248ad3'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:17.748923 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-2dbbe0b5-8e14-41b3-bf03-b07638248ad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f4497b720094374b4c6cf083c29268c', 'request_id': 'req-2dbbe0b5-8e14-41b3-bf03-b07638248ad3'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:17.774370 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-2dbbe0b5-8e14-41b3-bf03-b07638248ad3 admin admin] The token cache is being invalidate because project 8f4497b720094374b4c6cf083c29268c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:09:17.774790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-2dbbe0b5-8e14-41b3-bf03-b07638248ad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:17.775691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-2dbbe0b5-8e14-41b3-bf03-b07638248ad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:17.778509 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 958/1873] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:09:17 2022] DELETE /identity/v3/projects/8f4497b720094374b4c6cf083c29268c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:09:17.784030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1a74f-300b-4d22-af8a-0797604069f5 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:17.785440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1a74f-300b-4d22-af8a-0797604069f5 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:17.785440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1a74f-300b-4d22-af8a-0797604069f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:17.811843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-a7c1a74f-300b-4d22-af8a-0797604069f5 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:17.812545 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 916/1874] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:17.813315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a7c1a74f-300b-4d22-af8a-0797604069f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:17.829102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011c22a-c6e1-4a94-8916-c38bf63c4683 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:17.829313 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 959/1875] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:17.829433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c011c22a-c6e1-4a94-8916-c38bf63c4683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:17.832220 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b58b43-67f2-4aa6-a88c-782438c3aecc None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:17.832653 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b58b43-67f2-4aa6-a88c-782438c3aecc None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:17.833022 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b58b43-67f2-4aa6-a88c-782438c3aecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:17.856308 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-87b58b43-67f2-4aa6-a88c-782438c3aecc None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:17.857649 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-87b58b43-67f2-4aa6-a88c-782438c3aecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:17.870278 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 960/1876] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:17.876217 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b58b43-67f2-4aa6-a88c-782438c3aecc None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:17.890709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b58b43-67f2-4aa6-a88c-782438c3aecc None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:17.896024 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330624c-6578-4aaa-946f-7fc63c5ac526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:17.896635 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330624c-6578-4aaa-946f-7fc63c5ac526 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:17.896913 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330624c-6578-4aaa-946f-7fc63c5ac526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:17.897406 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330624c-6578-4aaa-946f-7fc63c5ac526 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:17.898155 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330624c-6578-4aaa-946f-7fc63c5ac526 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:09:17.898460 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330624c-6578-4aaa-946f-7fc63c5ac526 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:17.900313 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330624c-6578-4aaa-946f-7fc63c5ac526 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:17.910317 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011c22a-c6e1-4a94-8916-c38bf63c4683 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:17.918231 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 961/1877] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:09:17.919109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efa7bc-49b6-49f9-bca6-3cb395b3da97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:17.919247 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330624c-6578-4aaa-946f-7fc63c5ac526 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:17.920316 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efa7bc-49b6-49f9-bca6-3cb395b3da97 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:17.920766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efa7bc-49b6-49f9-bca6-3cb395b3da97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:17.921202 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efa7bc-49b6-49f9-bca6-3cb395b3da97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:17.931976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330624c-6578-4aaa-946f-7fc63c5ac526 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:17.933061 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66efa7bc-49b6-49f9-bca6-3cb395b3da97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:17.934646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66efa7bc-49b6-49f9-bca6-3cb395b3da97 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:17.938961 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20fff8c-4e53-40a5-aac5-90e91aa7629f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:17.939602 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fff8c-4e53-40a5-aac5-90e91aa7629f admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:17.940385 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fff8c-4e53-40a5-aac5-90e91aa7629f admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:17.943306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20fff8c-4e53-40a5-aac5-90e91aa7629f admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:17.944437 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20fff8c-4e53-40a5-aac5-90e91aa7629f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:17.949233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20fff8c-4e53-40a5-aac5-90e91aa7629f admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:17.952265 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 917/1878] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:09:17.996390 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 962/1879] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:09:17 2022] POST /identity/v3/projects => generated 359 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:09:17.997699 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efa7bc-49b6-49f9-bca6-3cb395b3da97 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:18.011995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efa7bc-49b6-49f9-bca6-3cb395b3da97 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:18.106682 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972eba5-41f9-464d-a618-cf763773ed95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:18.107910 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972eba5-41f9-464d-a618-cf763773ed95 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:18.108555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972eba5-41f9-464d-a618-cf763773ed95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:18.108983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972eba5-41f9-464d-a618-cf763773ed95 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:18.109772 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972eba5-41f9-464d-a618-cf763773ed95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:18.112770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972eba5-41f9-464d-a618-cf763773ed95 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:18.160807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20fff8c-4e53-40a5-aac5-90e91aa7629f admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:18.161716 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 918/1880] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:09:17 2022] POST /identity/v3/users => generated 280 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:09:18.177637 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20fff8c-4e53-40a5-aac5-90e91aa7629f admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:18.191213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3982a8-1272-45a1-bd5e-196a2b7c34ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:18.192104 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3982a8-1272-45a1-bd5e-196a2b7c34ea admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:18.192537 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3982a8-1272-45a1-bd5e-196a2b7c34ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:18.192967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3982a8-1272-45a1-bd5e-196a2b7c34ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:18.193792 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3982a8-1272-45a1-bd5e-196a2b7c34ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:18.195668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3982a8-1272-45a1-bd5e-196a2b7c34ea admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:18.220770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 963/1881] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:09:18.224678 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972eba5-41f9-464d-a618-cf763773ed95 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:18.239735 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972eba5-41f9-464d-a618-cf763773ed95 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:18.248933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e888b-1e56-4246-bc88-0c9a72c82dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:18.253656 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e888b-1e56-4246-bc88-0c9a72c82dba admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:18.255669 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e888b-1e56-4246-bc88-0c9a72c82dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:18.255986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e888b-1e56-4246-bc88-0c9a72c82dba admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:18.256778 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e888b-1e56-4246-bc88-0c9a72c82dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:18.262525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e888b-1e56-4246-bc88-0c9a72c82dba admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:18.290325 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3982a8-1272-45a1-bd5e-196a2b7c34ea admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:18.291113 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 919/1882] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:09:18.312990 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3982a8-1272-45a1-bd5e-196a2b7c34ea admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:18.321170 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56876016-4a06-40b1-ad48-da1f9b05431f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:18.322291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56876016-4a06-40b1-ad48-da1f9b05431f admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:18.322718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56876016-4a06-40b1-ad48-da1f9b05431f admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:18.323327 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56876016-4a06-40b1-ad48-da1f9b05431f admin admin] PATH_INFO: `/v3/projects/7b92db0755214bf781ad31eaa026a37d/users/d25b8d4428364f33b1e256a7e7dc6dfc/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:18.371351 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56876016-4a06-40b1-ad48-da1f9b05431f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b92db0755214bf781ad31eaa026a37d, user_id=d25b8d4428364f33b1e256a7e7dc6dfc, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:18.374212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56876016-4a06-40b1-ad48-da1f9b05431f admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:18.404772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 964/1883] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:09:18 2022] PUT /identity/v3/projects/7b92db0755214bf781ad31eaa026a37d/users/d25b8d4428364f33b1e256a7e7dc6dfc/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:09:18.406527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813be057-dbac-4b9d-b806-ff3b55cbcef0 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:18.406814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813be057-dbac-4b9d-b806-ff3b55cbcef0 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:18.407131 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813be057-dbac-4b9d-b806-ff3b55cbcef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:18.472079 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-813be057-dbac-4b9d-b806-ff3b55cbcef0 None None] MFA Rules not processed for user `d25b8d4428364f33b1e256a7e7dc6dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:18.473337 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-813be057-dbac-4b9d-b806-ff3b55cbcef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:18.514490 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:18.591059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 920/1884] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:18 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:18.632438 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56876016-4a06-40b1-ad48-da1f9b05431f admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:18.640965 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-56876016-4a06-40b1-ad48-da1f9b05431f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:18.767657 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56876016-4a06-40b1-ad48-da1f9b05431f admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:18.780405 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb69355-96da-47c3-a781-37c51bec6ff9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:18.782687 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb69355-96da-47c3-a781-37c51bec6ff9 service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:18.783830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb69355-96da-47c3-a781-37c51bec6ff9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:18.784575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb69355-96da-47c3-a781-37c51bec6ff9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:18.801848 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb69355-96da-47c3-a781-37c51bec6ff9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:18.803584 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb69355-96da-47c3-a781-37c51bec6ff9 service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:18.811177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:18.825495 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 965/1885] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:18 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:09:18.842833 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813be057-dbac-4b9d-b806-ff3b55cbcef0 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:18.969019 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813be057-dbac-4b9d-b806-ff3b55cbcef0 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:18.972523 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb69355-96da-47c3-a781-37c51bec6ff9 service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:18.974022 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-fdb69355-96da-47c3-a781-37c51bec6ff9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:18.980180 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f2ac6e-7fe6-4cd0-a6de-3d63b0bcc23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:18.980976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2ac6e-7fe6-4cd0-a6de-3d63b0bcc23d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:18.981699 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2ac6e-7fe6-4cd0-a6de-3d63b0bcc23d admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:18.982071 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f2ac6e-7fe6-4cd0-a6de-3d63b0bcc23d admin admin] PATH_INFO: `/v3/users/d25b8d4428364f33b1e256a7e7dc6dfc` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:18.983541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2ac6e-7fe6-4cd0-a6de-3d63b0bcc23d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d25b8d4428364f33b1e256a7e7dc6dfc)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:18.986757 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f2ac6e-7fe6-4cd0-a6de-3d63b0bcc23d admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:19.008371 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb69355-96da-47c3-a781-37c51bec6ff9 service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:19.016985 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d4d06-9a6a-434e-acfe-afb0deb86ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:19.018556 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d4d06-9a6a-434e-acfe-afb0deb86ecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:19.018726 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d4d06-9a6a-434e-acfe-afb0deb86ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:19.019230 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d4d06-9a6a-434e-acfe-afb0deb86ecf admin admin] PATH_INFO: `/v3/projects/606a7f5c127742ca809573202fb8bdf8` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:19.030263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d4d06-9a6a-434e-acfe-afb0deb86ecf admin admin] RBAC: Authorizing `identity:delete_project(project_id=606a7f5c127742ca809573202fb8bdf8)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:19.033352 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d4d06-9a6a-434e-acfe-afb0deb86ecf admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:19.086335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-82f2ac6e-7fe6-4cd0-a6de-3d63b0bcc23d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd25b8d4428364f33b1e256a7e7dc6dfc'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:19.087919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-82f2ac6e-7fe6-4cd0-a6de-3d63b0bcc23d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd25b8d4428364f33b1e256a7e7dc6dfc'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:19.101729 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-82f2ac6e-7fe6-4cd0-a6de-3d63b0bcc23d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd25b8d4428364f33b1e256a7e7dc6dfc'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:19.110839 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ad9d4d06-9a6a-434e-acfe-afb0deb86ecf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '606a7f5c127742ca809573202fb8bdf8', 'request_id': 'req-ad9d4d06-9a6a-434e-acfe-afb0deb86ecf'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:19.111617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-82f2ac6e-7fe6-4cd0-a6de-3d63b0bcc23d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd25b8d4428364f33b1e256a7e7dc6dfc'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:19.112516 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-82f2ac6e-7fe6-4cd0-a6de-3d63b0bcc23d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd25b8d4428364f33b1e256a7e7dc6dfc'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:19.120107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ad9d4d06-9a6a-434e-acfe-afb0deb86ecf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '606a7f5c127742ca809573202fb8bdf8', 'request_id': 'req-ad9d4d06-9a6a-434e-acfe-afb0deb86ecf'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:19.126722 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ad9d4d06-9a6a-434e-acfe-afb0deb86ecf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '606a7f5c127742ca809573202fb8bdf8', 'request_id': 'req-ad9d4d06-9a6a-434e-acfe-afb0deb86ecf'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:19.129186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 921/1886] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:09:18 2022] DELETE /identity/v3/users/d25b8d4428364f33b1e256a7e7dc6dfc => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:19.133573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a947297-ced3-4a6a-a689-9668a85e99e3 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:19.134044 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a947297-ced3-4a6a-a689-9668a85e99e3 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:19.134464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a947297-ced3-4a6a-a689-9668a85e99e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:19.142233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ad9d4d06-9a6a-434e-acfe-afb0deb86ecf admin admin] The token cache is being invalidate because project 606a7f5c127742ca809573202fb8bdf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:09:19.142524 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ad9d4d06-9a6a-434e-acfe-afb0deb86ecf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:19.143361 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ad9d4d06-9a6a-434e-acfe-afb0deb86ecf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:19.145886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 966/1887] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:09:18 2022] DELETE /identity/v3/projects/606a7f5c127742ca809573202fb8bdf8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:09:19.150295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d4d06-9a6a-434e-acfe-afb0deb86ecf admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:19.151714 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-ad9d4d06-9a6a-434e-acfe-afb0deb86ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:19.156017 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-6a947297-ced3-4a6a-a689-9668a85e99e3 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:19.158030 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6a947297-ced3-4a6a-a689-9668a85e99e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:19.206810 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:19.231653 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d4d06-9a6a-434e-acfe-afb0deb86ecf admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:19.291429 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 922/1888] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:19.299791 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a947297-ced3-4a6a-a689-9668a85e99e3 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:19.301193 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6a947297-ced3-4a6a-a689-9668a85e99e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:19.331528 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac1172-649d-49f6-8278-39efceedd960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:19.332772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac1172-649d-49f6-8278-39efceedd960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:19.333537 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac1172-649d-49f6-8278-39efceedd960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:19.333815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac1172-649d-49f6-8278-39efceedd960 admin admin] PATH_INFO: `/v3/users/06188fe0f2f44142a3d3f1a3bf859faf` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:19.335015 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac1172-649d-49f6-8278-39efceedd960 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06188fe0f2f44142a3d3f1a3bf859faf)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:19.336997 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac1172-649d-49f6-8278-39efceedd960 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:19.364141 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a947297-ced3-4a6a-a689-9668a85e99e3 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:19.368130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c747a4-9bec-4786-af38-853b4d61e61d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:19.368846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c747a4-9bec-4786-af38-853b4d61e61d service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:19.369190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c747a4-9bec-4786-af38-853b4d61e61d service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:19.369521 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c747a4-9bec-4786-af38-853b4d61e61d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:19.381751 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c747a4-9bec-4786-af38-853b4d61e61d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:19.383833 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c747a4-9bec-4786-af38-853b4d61e61d service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:19.385091 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:19.397345 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 923/1889] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:09:19.404032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-60ac1172-649d-49f6-8278-39efceedd960 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06188fe0f2f44142a3d3f1a3bf859faf'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:19.413234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-60ac1172-649d-49f6-8278-39efceedd960 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06188fe0f2f44142a3d3f1a3bf859faf'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:19.421683 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-60ac1172-649d-49f6-8278-39efceedd960 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06188fe0f2f44142a3d3f1a3bf859faf'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:19.422415 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-60ac1172-649d-49f6-8278-39efceedd960 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06188fe0f2f44142a3d3f1a3bf859faf'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:19.424090 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-60ac1172-649d-49f6-8278-39efceedd960 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06188fe0f2f44142a3d3f1a3bf859faf'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:19.447151 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 967/1890] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:09:19 2022] DELETE /identity/v3/users/06188fe0f2f44142a3d3f1a3bf859faf => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:09:19.865862 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c747a4-9bec-4786-af38-853b4d61e61d service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:19.881183 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c747a4-9bec-4786-af38-853b4d61e61d service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:19.914382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5c03c-096c-4d18-8923-baf6d7cc378d None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:19.914993 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5c03c-096c-4d18-8923-baf6d7cc378d None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:19.915837 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff5c03c-096c-4d18-8923-baf6d7cc378d None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:19.917863 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 968/1891] 10.222.0.36 () {54 vars in 921 bytes} [Fri Sep 2 09:09:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:09:19.937321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfead0e-dd24-4a69-9c29-d00fa22f8d09 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:19.937662 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfead0e-dd24-4a69-9c29-d00fa22f8d09 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:19.937993 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfead0e-dd24-4a69-9c29-d00fa22f8d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:19.969011 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-1dfead0e-dd24-4a69-9c29-d00fa22f8d09 None None] MFA Rules not processed for user `cce502a83f5343558c2136c0193e7d85`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:19.971902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-1dfead0e-dd24-4a69-9c29-d00fa22f8d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:20.023409 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66064c-fc63-4770-acab-bfa9a5ecae0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:20.024434 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66064c-fc63-4770-acab-bfa9a5ecae0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:20.024813 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66064c-fc63-4770-acab-bfa9a5ecae0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:20.025635 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66064c-fc63-4770-acab-bfa9a5ecae0a admin admin] PATH_INFO: `/v3/projects/4f26863d92414fa2b7522dabe0d1bdb3` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:20.026869 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66064c-fc63-4770-acab-bfa9a5ecae0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f26863d92414fa2b7522dabe0d1bdb3)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:20.029083 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66064c-fc63-4770-acab-bfa9a5ecae0a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:20.066174 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 969/1892] 10.222.0.36 () {58 vars in 1018 bytes} [Fri Sep 2 09:09:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:20.097939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7c66064c-fc63-4770-acab-bfa9a5ecae0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f26863d92414fa2b7522dabe0d1bdb3', 'request_id': 'req-7c66064c-fc63-4770-acab-bfa9a5ecae0a'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:20.107349 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7c66064c-fc63-4770-acab-bfa9a5ecae0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f26863d92414fa2b7522dabe0d1bdb3', 'request_id': 'req-7c66064c-fc63-4770-acab-bfa9a5ecae0a'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:20.114446 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7c66064c-fc63-4770-acab-bfa9a5ecae0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f26863d92414fa2b7522dabe0d1bdb3', 'request_id': 'req-7c66064c-fc63-4770-acab-bfa9a5ecae0a'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:20.131405 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7c66064c-fc63-4770-acab-bfa9a5ecae0a admin admin] The token cache is being invalidate because project 4f26863d92414fa2b7522dabe0d1bdb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:09:20.131797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7c66064c-fc63-4770-acab-bfa9a5ecae0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:20.132439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7c66064c-fc63-4770-acab-bfa9a5ecae0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:20.134317 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 924/1893] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:09:19 2022] DELETE /identity/v3/projects/4f26863d92414fa2b7522dabe0d1bdb3 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:20.139404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c8f1e-8623-4500-bf79-f379746e6fa5 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:20.139917 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c8f1e-8623-4500-bf79-f379746e6fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:20.140375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c8f1e-8623-4500-bf79-f379746e6fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:20.162805 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-b08c8f1e-8623-4500-bf79-f379746e6fa5 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:20.164389 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-b08c8f1e-8623-4500-bf79-f379746e6fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:20.269131 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66064c-fc63-4770-acab-bfa9a5ecae0a admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:20.271926 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7c66064c-fc63-4770-acab-bfa9a5ecae0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:20.272077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 970/1894] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:20.273947 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c45ba3-e586-4198-aeda-d8d0355cfd0b None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:20.274544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c45ba3-e586-4198-aeda-d8d0355cfd0b None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:20.275023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c45ba3-e586-4198-aeda-d8d0355cfd0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:20.299495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66064c-fc63-4770-acab-bfa9a5ecae0a admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:20.309149 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-92c45ba3-e586-4198-aeda-d8d0355cfd0b None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:20.313046 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-92c45ba3-e586-4198-aeda-d8d0355cfd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:20.327093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997eaaa-9493-45bc-a348-1ac6f42095de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:20.327093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997eaaa-9493-45bc-a348-1ac6f42095de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:20.327093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997eaaa-9493-45bc-a348-1ac6f42095de admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:20.327093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d997eaaa-9493-45bc-a348-1ac6f42095de admin admin] PATH_INFO: `/v3/projects/7b92db0755214bf781ad31eaa026a37d` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:20.327581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997eaaa-9493-45bc-a348-1ac6f42095de admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b92db0755214bf781ad31eaa026a37d)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:20.327581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d997eaaa-9493-45bc-a348-1ac6f42095de admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:20.337591 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:20.343553 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 971/1895] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:20.345915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c45ba3-e586-4198-aeda-d8d0355cfd0b None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:20.355974 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-92c45ba3-e586-4198-aeda-d8d0355cfd0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:20.410112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:20.436366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d997eaaa-9493-45bc-a348-1ac6f42095de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b92db0755214bf781ad31eaa026a37d', 'request_id': 'req-d997eaaa-9493-45bc-a348-1ac6f42095de'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:20.446591 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d997eaaa-9493-45bc-a348-1ac6f42095de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b92db0755214bf781ad31eaa026a37d', 'request_id': 'req-d997eaaa-9493-45bc-a348-1ac6f42095de'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:20.454944 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d997eaaa-9493-45bc-a348-1ac6f42095de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b92db0755214bf781ad31eaa026a37d', 'request_id': 'req-d997eaaa-9493-45bc-a348-1ac6f42095de'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:20.471039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c45ba3-e586-4198-aeda-d8d0355cfd0b None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:20.471284 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d997eaaa-9493-45bc-a348-1ac6f42095de admin admin] The token cache is being invalidate because project 7b92db0755214bf781ad31eaa026a37d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:09:20.471384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d997eaaa-9493-45bc-a348-1ac6f42095de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:20.472009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d997eaaa-9493-45bc-a348-1ac6f42095de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:20.473954 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 925/1896] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:09:20 2022] DELETE /identity/v3/projects/7b92db0755214bf781ad31eaa026a37d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:20.474829 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997eaaa-9493-45bc-a348-1ac6f42095de admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:20.478680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-d997eaaa-9493-45bc-a348-1ac6f42095de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:20.568932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e06038-592f-4032-affe-7063c10149eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:20.571102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e06038-592f-4032-affe-7063c10149eb service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:20.572464 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e06038-592f-4032-affe-7063c10149eb service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:20.573346 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e06038-592f-4032-affe-7063c10149eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:20.575428 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-79e06038-592f-4032-affe-7063c10149eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:20.577782 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d997eaaa-9493-45bc-a348-1ac6f42095de admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:20.585553 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53acc7cf-a8fa-4c6e-aabc-3bcc207ebf87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:20.586635 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acc7cf-a8fa-4c6e-aabc-3bcc207ebf87 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:20.587511 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acc7cf-a8fa-4c6e-aabc-3bcc207ebf87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:20.587998 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acc7cf-a8fa-4c6e-aabc-3bcc207ebf87 admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:20.589096 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53acc7cf-a8fa-4c6e-aabc-3bcc207ebf87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:09:20.589790 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53acc7cf-a8fa-4c6e-aabc-3bcc207ebf87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:20.591775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53acc7cf-a8fa-4c6e-aabc-3bcc207ebf87 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:20.592208 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e06038-592f-4032-affe-7063c10149eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:20.593811 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e06038-592f-4032-affe-7063c10149eb service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:20.608495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 926/1897] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:09:20.608495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4303979-2a5b-49b9-a767-e65946fd9a2c None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:20.608495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4303979-2a5b-49b9-a767-e65946fd9a2c None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:20.608495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4303979-2a5b-49b9-a767-e65946fd9a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:20.618961 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 972/1898] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:09:20.621656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e06038-592f-4032-affe-7063c10149eb service nova] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:20.631225 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-a4303979-2a5b-49b9-a767-e65946fd9a2c None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:20.632444 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-a4303979-2a5b-49b9-a767-e65946fd9a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:20.641397 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 927/1899] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:20.649862 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e06038-592f-4032-affe-7063c10149eb service nova] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:20.649990 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277587f-e220-4cf9-9b98-0927ecf3d187 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:20.650255 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277587f-e220-4cf9-9b98-0927ecf3d187 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:20.650639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277587f-e220-4cf9-9b98-0927ecf3d187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:20.655344 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32348f4b-89c9-48ce-a463-58ffc152ac0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:20.655920 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32348f4b-89c9-48ce-a463-58ffc152ac0c admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:20.656205 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32348f4b-89c9-48ce-a463-58ffc152ac0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:20.656482 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32348f4b-89c9-48ce-a463-58ffc152ac0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:20.657153 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32348f4b-89c9-48ce-a463-58ffc152ac0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:20.659099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32348f4b-89c9-48ce-a463-58ffc152ac0c admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:20.676317 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-6277587f-e220-4cf9-9b98-0927ecf3d187 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:20.678496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6277587f-e220-4cf9-9b98-0927ecf3d187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:20.689868 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 928/1900] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:20.708708 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277587f-e220-4cf9-9b98-0927ecf3d187 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:20.709886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 973/1901] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:09:20 2022] POST /identity/v3/projects => generated 385 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:09:20.710856 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32348f4b-89c9-48ce-a463-58ffc152ac0c admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:20.721657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277587f-e220-4cf9-9b98-0927ecf3d187 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:20.798879 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32348f4b-89c9-48ce-a463-58ffc152ac0c admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:20.804148 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c928696-a10f-48d9-8b01-5432d2e49bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:20.804990 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c928696-a10f-48d9-8b01-5432d2e49bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:20.805343 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c928696-a10f-48d9-8b01-5432d2e49bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:20.805691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c928696-a10f-48d9-8b01-5432d2e49bb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:20.806470 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c928696-a10f-48d9-8b01-5432d2e49bb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:20.808262 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e9a0f-03d9-453a-9539-4b7ab1cc93d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:20.809175 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c928696-a10f-48d9-8b01-5432d2e49bb5 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:20.809294 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e9a0f-03d9-453a-9539-4b7ab1cc93d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:20.809294 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e9a0f-03d9-453a-9539-4b7ab1cc93d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:20.809444 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e9a0f-03d9-453a-9539-4b7ab1cc93d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:20.810298 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e9a0f-03d9-453a-9539-4b7ab1cc93d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:09:20.810634 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e9a0f-03d9-453a-9539-4b7ab1cc93d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:20.812625 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e9a0f-03d9-453a-9539-4b7ab1cc93d3 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:20.829638 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 929/1902] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:09:20.832427 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 974/1903] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:09:20 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:09:20.836609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e9a0f-03d9-453a-9539-4b7ab1cc93d3 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:20.836901 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c928696-a10f-48d9-8b01-5432d2e49bb5 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:20.848793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c928696-a10f-48d9-8b01-5432d2e49bb5 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:20.849274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e9a0f-03d9-453a-9539-4b7ab1cc93d3 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:20.853519 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b5f01-4548-4cf4-b51a-671da9b385dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:20.853979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d5903-3734-4ccd-82ab-a1c973a740fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:20.854182 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b5f01-4548-4cf4-b51a-671da9b385dd admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:20.854391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b5f01-4548-4cf4-b51a-671da9b385dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:20.854578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d5903-3734-4ccd-82ab-a1c973a740fc admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:20.854683 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b5f01-4548-4cf4-b51a-671da9b385dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:20.854856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d5903-3734-4ccd-82ab-a1c973a740fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:20.855203 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d5903-3734-4ccd-82ab-a1c973a740fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:20.855506 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b5f01-4548-4cf4-b51a-671da9b385dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:20.855978 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d5903-3734-4ccd-82ab-a1c973a740fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:20.857239 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b5f01-4548-4cf4-b51a-671da9b385dd admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:20.857847 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d5903-3734-4ccd-82ab-a1c973a740fc admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:20.868874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 975/1904] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:09:20.872831 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b5f01-4548-4cf4-b51a-671da9b385dd admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:20.873230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:20.885555 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b5f01-4548-4cf4-b51a-671da9b385dd admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:20.892569 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b7827-e907-4772-8b4d-2a9ea2883802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:20.896261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b7827-e907-4772-8b4d-2a9ea2883802 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:20.896661 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b7827-e907-4772-8b4d-2a9ea2883802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:20.897070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b7827-e907-4772-8b4d-2a9ea2883802 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:20.900987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b7827-e907-4772-8b4d-2a9ea2883802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:20.901317 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 930/1905] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:09:20 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:09:20.901407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b7827-e907-4772-8b4d-2a9ea2883802 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:20.908242 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d5903-3734-4ccd-82ab-a1c973a740fc admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:20.913765 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 976/1906] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:09:20.920778 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b7827-e907-4772-8b4d-2a9ea2883802 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:20.926567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:20.939233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b7827-e907-4772-8b4d-2a9ea2883802 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:20.991357 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d5903-3734-4ccd-82ab-a1c973a740fc admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:20.998021 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baadb2a-ee79-4b2a-9a3c-3e21f4aeb1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:20.998706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baadb2a-ee79-4b2a-9a3c-3e21f4aeb1eb admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:20.999165 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baadb2a-ee79-4b2a-9a3c-3e21f4aeb1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:20.999500 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baadb2a-ee79-4b2a-9a3c-3e21f4aeb1eb admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:21.001512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baadb2a-ee79-4b2a-9a3c-3e21f4aeb1eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:21.003687 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baadb2a-ee79-4b2a-9a3c-3e21f4aeb1eb admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:21.032234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 931/1907] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:09:20 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:09:21.037266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5a85c-5316-4343-a634-563a6ff6ee93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:21.038326 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5a85c-5316-4343-a634-563a6ff6ee93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:21.039442 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baadb2a-ee79-4b2a-9a3c-3e21f4aeb1eb admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:21.040430 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5a85c-5316-4343-a634-563a6ff6ee93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:21.046515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5a85c-5316-4343-a634-563a6ff6ee93 admin admin] PATH_INFO: `/v3/projects/d949d34ff8484dbd899fda5209c618a9/users/048e5008adaa44b3a5cf5a9a52994dde/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:21.055773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baadb2a-ee79-4b2a-9a3c-3e21f4aeb1eb admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:21.068067 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd6f81-8b53-4ab2-b615-c701746e1598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:21.068719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5a85c-5316-4343-a634-563a6ff6ee93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d949d34ff8484dbd899fda5209c618a9, user_id=048e5008adaa44b3a5cf5a9a52994dde, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:21.069224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd6f81-8b53-4ab2-b615-c701746e1598 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:21.069224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd6f81-8b53-4ab2-b615-c701746e1598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:21.070088 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd6f81-8b53-4ab2-b615-c701746e1598 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:21.071169 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd6f81-8b53-4ab2-b615-c701746e1598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:21.073415 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd6f81-8b53-4ab2-b615-c701746e1598 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:21.074175 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a5a85c-5316-4343-a634-563a6ff6ee93 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:21.090225 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 932/1908] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:09:21.091373 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd6f81-8b53-4ab2-b615-c701746e1598 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:21.099714 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 977/1909] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:09:20 2022] PUT /identity/v3/projects/d949d34ff8484dbd899fda5209c618a9/users/048e5008adaa44b3a5cf5a9a52994dde/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:21.108607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd6f81-8b53-4ab2-b615-c701746e1598 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:21.111360 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733254b2-abe1-4af9-9685-fb69127f90d0 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:21.111650 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733254b2-abe1-4af9-9685-fb69127f90d0 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:21.111948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733254b2-abe1-4af9-9685-fb69127f90d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:21.164466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-733254b2-abe1-4af9-9685-fb69127f90d0 None None] MFA Rules not processed for user `048e5008adaa44b3a5cf5a9a52994dde`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:21.165856 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-733254b2-abe1-4af9-9685-fb69127f90d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:21.183908 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3f4a3-5e00-4c74-a519-d8a7bdffbacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:21.186449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3f4a3-5e00-4c74-a519-d8a7bdffbacf admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:21.186745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3f4a3-5e00-4c74-a519-d8a7bdffbacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:21.187025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3f4a3-5e00-4c74-a519-d8a7bdffbacf admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:21.188060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3f4a3-5e00-4c74-a519-d8a7bdffbacf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:21.190497 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3f4a3-5e00-4c74-a519-d8a7bdffbacf admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:21.206577 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 933/1910] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:09:21.211346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3f4a3-5e00-4c74-a519-d8a7bdffbacf admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:21.237871 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3f4a3-5e00-4c74-a519-d8a7bdffbacf admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:21.250263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30919ae8-516f-4d2d-a5e7-e1e2107032c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:21.251010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30919ae8-516f-4d2d-a5e7-e1e2107032c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:21.251806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30919ae8-516f-4d2d-a5e7-e1e2107032c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:21.252082 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30919ae8-516f-4d2d-a5e7-e1e2107032c4 admin admin] PATH_INFO: `/v3/projects/90626527642f465990ec7b1f2fc9078d/users/97ba745370674d858e93ae8ee5145730/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:21.274131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 978/1911] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:21.289493 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733254b2-abe1-4af9-9685-fb69127f90d0 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:21.290906 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30919ae8-516f-4d2d-a5e7-e1e2107032c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90626527642f465990ec7b1f2fc9078d, user_id=97ba745370674d858e93ae8ee5145730, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:21.291051 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-733254b2-abe1-4af9-9685-fb69127f90d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:21.297187 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30919ae8-516f-4d2d-a5e7-e1e2107032c4 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:21.316329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 934/1912] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:09:21 2022] PUT /identity/v3/projects/90626527642f465990ec7b1f2fc9078d/users/97ba745370674d858e93ae8ee5145730/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:09:21.326731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1c184-16a1-4863-b7bd-c5f2ff0f0456 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:21.327424 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1c184-16a1-4863-b7bd-c5f2ff0f0456 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:21.327849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf1c184-16a1-4863-b7bd-c5f2ff0f0456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:21.377554 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733254b2-abe1-4af9-9685-fb69127f90d0 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:21.392266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-8cf1c184-16a1-4863-b7bd-c5f2ff0f0456 None None] MFA Rules not processed for user `97ba745370674d858e93ae8ee5145730`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:21.393683 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-8cf1c184-16a1-4863-b7bd-c5f2ff0f0456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:21.441736 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:21.450493 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:21.484424 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a989c-b816-4965-9359-919b3baa9721 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:21.485143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a989c-b816-4965-9359-919b3baa9721 service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:21.486276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a989c-b816-4965-9359-919b3baa9721 service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:21.486607 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a989c-b816-4965-9359-919b3baa9721 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:21.502329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 935/1913] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:09:21.514095 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1c184-16a1-4863-b7bd-c5f2ff0f0456 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:21.515520 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-8cf1c184-16a1-4863-b7bd-c5f2ff0f0456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:21.537322 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf1c184-16a1-4863-b7bd-c5f2ff0f0456 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:21.554305 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c6b9d4-beba-4502-96bf-e191f64fc0c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:21.555384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6b9d4-beba-4502-96bf-e191f64fc0c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:21.556014 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6b9d4-beba-4502-96bf-e191f64fc0c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:21.556495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c6b9d4-beba-4502-96bf-e191f64fc0c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:21.561780 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a989c-b816-4965-9359-919b3baa9721 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:21.563956 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a989c-b816-4965-9359-919b3baa9721 service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:21.569706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6b9d4-beba-4502-96bf-e191f64fc0c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:21.571625 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c6b9d4-beba-4502-96bf-e191f64fc0c4 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:21.600832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 979/1914] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:09:21.612652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 936/1915] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:09:27.970127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a989c-b816-4965-9359-919b3baa9721 service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:28.034232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a989c-b816-4965-9359-919b3baa9721 service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:28.038733 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd44e6-497a-4774-aeda-5df9967a2b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:28.039393 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd44e6-497a-4774-aeda-5df9967a2b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:28.039679 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd44e6-497a-4774-aeda-5df9967a2b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:28.039950 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd44e6-497a-4774-aeda-5df9967a2b9e admin admin] PATH_INFO: `/v3/users/048e5008adaa44b3a5cf5a9a52994dde` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:28.041078 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd44e6-497a-4774-aeda-5df9967a2b9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=048e5008adaa44b3a5cf5a9a52994dde)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:28.043014 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd44e6-497a-4774-aeda-5df9967a2b9e admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:28.091840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-bffd44e6-497a-4774-aeda-5df9967a2b9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '048e5008adaa44b3a5cf5a9a52994dde'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:28.099053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-bffd44e6-497a-4774-aeda-5df9967a2b9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '048e5008adaa44b3a5cf5a9a52994dde'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:28.107300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-bffd44e6-497a-4774-aeda-5df9967a2b9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '048e5008adaa44b3a5cf5a9a52994dde'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:28.107943 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-bffd44e6-497a-4774-aeda-5df9967a2b9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '048e5008adaa44b3a5cf5a9a52994dde'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:28.108546 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-bffd44e6-497a-4774-aeda-5df9967a2b9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '048e5008adaa44b3a5cf5a9a52994dde'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:28.126713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 980/1916] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:09:27 2022] DELETE /identity/v3/users/048e5008adaa44b3a5cf5a9a52994dde => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:28.130915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01c764-8eb4-4803-a410-602b254d9e79 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:28.131426 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01c764-8eb4-4803-a410-602b254d9e79 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:28.131521 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01c764-8eb4-4803-a410-602b254d9e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:28.150420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-9b01c764-8eb4-4803-a410-602b254d9e79 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:28.151637 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-9b01c764-8eb4-4803-a410-602b254d9e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:28.209860 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 937/1917] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:28.219556 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd44e6-497a-4774-aeda-5df9967a2b9e admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:28.221490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-bffd44e6-497a-4774-aeda-5df9967a2b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:28.227988 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:28.281260 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd44e6-497a-4774-aeda-5df9967a2b9e admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:28.284986 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d265469-b8a2-4f8e-acbd-0ea357687f12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:28.285594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d265469-b8a2-4f8e-acbd-0ea357687f12 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:28.285874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d265469-b8a2-4f8e-acbd-0ea357687f12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:28.286149 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d265469-b8a2-4f8e-acbd-0ea357687f12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:28.295374 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d265469-b8a2-4f8e-acbd-0ea357687f12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:28.296799 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d265469-b8a2-4f8e-acbd-0ea357687f12 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:28.306437 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 981/1918] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:09:28.584203 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01c764-8eb4-4803-a410-602b254d9e79 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:28.585701 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-9b01c764-8eb4-4803-a410-602b254d9e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:28.600004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01c764-8eb4-4803-a410-602b254d9e79 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:28.604874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd94e8-8826-42a9-88e2-3c396b30d515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:28.605567 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd94e8-8826-42a9-88e2-3c396b30d515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:28.605850 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd94e8-8826-42a9-88e2-3c396b30d515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:28.606119 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd94e8-8826-42a9-88e2-3c396b30d515 admin admin] PATH_INFO: `/v3/projects/d949d34ff8484dbd899fda5209c618a9` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:28.608570 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd94e8-8826-42a9-88e2-3c396b30d515 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d949d34ff8484dbd899fda5209c618a9)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:28.619237 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bd94e8-8826-42a9-88e2-3c396b30d515 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:28.670501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-33bd94e8-8826-42a9-88e2-3c396b30d515 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd949d34ff8484dbd899fda5209c618a9', 'request_id': 'req-33bd94e8-8826-42a9-88e2-3c396b30d515'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:28.678097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-33bd94e8-8826-42a9-88e2-3c396b30d515 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd949d34ff8484dbd899fda5209c618a9', 'request_id': 'req-33bd94e8-8826-42a9-88e2-3c396b30d515'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:28.683567 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-33bd94e8-8826-42a9-88e2-3c396b30d515 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd949d34ff8484dbd899fda5209c618a9', 'request_id': 'req-33bd94e8-8826-42a9-88e2-3c396b30d515'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:28.700359 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-33bd94e8-8826-42a9-88e2-3c396b30d515 admin admin] The token cache is being invalidate because project d949d34ff8484dbd899fda5209c618a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:09:28.700639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-33bd94e8-8826-42a9-88e2-3c396b30d515 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:28.701342 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-33bd94e8-8826-42a9-88e2-3c396b30d515 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:28.703460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 938/1919] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:09:28 2022] DELETE /identity/v3/projects/d949d34ff8484dbd899fda5209c618a9 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:28.708509 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717b947-aa6e-45ad-afeb-77aa5f22e72c None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:28.708791 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717b947-aa6e-45ad-afeb-77aa5f22e72c None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:28.709062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717b947-aa6e-45ad-afeb-77aa5f22e72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:28.728974 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-5717b947-aa6e-45ad-afeb-77aa5f22e72c None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:28.730323 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5717b947-aa6e-45ad-afeb-77aa5f22e72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:28.792383 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 982/1920] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:28.800559 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657124d-1e01-41f0-a621-b52842da2dcd None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:28.800849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657124d-1e01-41f0-a621-b52842da2dcd None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:28.801125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1657124d-1e01-41f0-a621-b52842da2dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:28.820527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-1657124d-1e01-41f0-a621-b52842da2dcd None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:28.821741 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1657124d-1e01-41f0-a621-b52842da2dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:28.830320 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 939/1921] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:28.834694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5717b947-aa6e-45ad-afeb-77aa5f22e72c None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:28.845750 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5717b947-aa6e-45ad-afeb-77aa5f22e72c None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:28.850872 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af5f07-8548-4692-a42d-87ae53034c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:28.851573 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af5f07-8548-4692-a42d-87ae53034c18 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:28.851919 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af5f07-8548-4692-a42d-87ae53034c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:28.852365 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af5f07-8548-4692-a42d-87ae53034c18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:28.853157 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af5f07-8548-4692-a42d-87ae53034c18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:09:28.853522 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af5f07-8548-4692-a42d-87ae53034c18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:28.855270 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2af5f07-8548-4692-a42d-87ae53034c18 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:28.868000 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 983/1922] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:09:28.872305 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657124d-1e01-41f0-a621-b52842da2dcd None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:28.882261 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1657124d-1e01-41f0-a621-b52842da2dcd None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:28.888734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a417f-a6e6-434d-9800-4d9c2448b50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:28.889463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a417f-a6e6-434d-9800-4d9c2448b50e admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:28.889739 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a417f-a6e6-434d-9800-4d9c2448b50e admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:28.890004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a417f-a6e6-434d-9800-4d9c2448b50e admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:28.890642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a417f-a6e6-434d-9800-4d9c2448b50e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:28.892609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a417f-a6e6-434d-9800-4d9c2448b50e admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:28.921374 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 940/1923] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:09:28 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:09:28.925685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af5f07-8548-4692-a42d-87ae53034c18 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:28.940310 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af5f07-8548-4692-a42d-87ae53034c18 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:29.006246 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85ab73-db09-456b-bab0-727069ca0e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:29.006941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85ab73-db09-456b-bab0-727069ca0e1c admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:29.007346 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85ab73-db09-456b-bab0-727069ca0e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:29.008574 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be85ab73-db09-456b-bab0-727069ca0e1c admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:29.009349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85ab73-db09-456b-bab0-727069ca0e1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:29.011454 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be85ab73-db09-456b-bab0-727069ca0e1c admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:29.043192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 984/1924] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:09:28 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:09:29.045314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a417f-a6e6-434d-9800-4d9c2448b50e admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:29.055446 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a417f-a6e6-434d-9800-4d9c2448b50e admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:29.060041 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2d185-1689-41a0-92cf-5abee7331da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:29.060722 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2d185-1689-41a0-92cf-5abee7331da2 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:29.061060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2d185-1689-41a0-92cf-5abee7331da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:29.061484 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2d185-1689-41a0-92cf-5abee7331da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:29.062133 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2d185-1689-41a0-92cf-5abee7331da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:29.064258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2d185-1689-41a0-92cf-5abee7331da2 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:29.069759 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:29.078160 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 941/1925] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:09:29.079735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85ab73-db09-456b-bab0-727069ca0e1c admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:29.089858 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be85ab73-db09-456b-bab0-727069ca0e1c admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:29.095211 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344053b6-5f43-4622-8bea-3cddae7b933e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:29.095865 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344053b6-5f43-4622-8bea-3cddae7b933e admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:29.096212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344053b6-5f43-4622-8bea-3cddae7b933e admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:29.096456 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344053b6-5f43-4622-8bea-3cddae7b933e admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:29.097093 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344053b6-5f43-4622-8bea-3cddae7b933e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:29.098741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344053b6-5f43-4622-8bea-3cddae7b933e admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:29.109974 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 985/1926] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:09:29.114168 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2d185-1689-41a0-92cf-5abee7331da2 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:29.124666 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2d185-1689-41a0-92cf-5abee7331da2 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:29.129903 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d2fb2-14ae-4c17-8e56-8d5a32941ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:29.131621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d2fb2-14ae-4c17-8e56-8d5a32941ec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:29.131621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d2fb2-14ae-4c17-8e56-8d5a32941ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:29.131621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d2fb2-14ae-4c17-8e56-8d5a32941ec8 admin admin] PATH_INFO: `/v3/projects/85a7c3487e204650adc69b55ef3feb66/users/e1daf1c8526245a9854f9b8672fa4222/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:29.146962 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d2fb2-14ae-4c17-8e56-8d5a32941ec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a7c3487e204650adc69b55ef3feb66, user_id=e1daf1c8526245a9854f9b8672fa4222, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:29.149853 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d2fb2-14ae-4c17-8e56-8d5a32941ec8 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:29.164407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 942/1927] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:09:29 2022] PUT /identity/v3/projects/85a7c3487e204650adc69b55ef3feb66/users/e1daf1c8526245a9854f9b8672fa4222/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:29.169483 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335fda7-80ab-4602-a28a-230d9c0906a7 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:29.169791 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335fda7-80ab-4602-a28a-230d9c0906a7 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:29.170062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335fda7-80ab-4602-a28a-230d9c0906a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:29.189713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-5335fda7-80ab-4602-a28a-230d9c0906a7 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:29.190907 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5335fda7-80ab-4602-a28a-230d9c0906a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:29.209848 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:29.255909 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 986/1928] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:29.265968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d2fb2-14ae-4c17-8e56-8d5a32941ec8 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:29.267719 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-352d2fb2-14ae-4c17-8e56-8d5a32941ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:29.324512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d2fb2-14ae-4c17-8e56-8d5a32941ec8 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:29.328197 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646d37e-a65e-4d63-a851-7ba328e41899 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:29.328802 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646d37e-a65e-4d63-a851-7ba328e41899 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:29.329077 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646d37e-a65e-4d63-a851-7ba328e41899 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:29.329347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646d37e-a65e-4d63-a851-7ba328e41899 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:29.338642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646d37e-a65e-4d63-a851-7ba328e41899 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:29.340106 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b646d37e-a65e-4d63-a851-7ba328e41899 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:29.350485 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 943/1929] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:09:30.559192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee9b07-2b7a-486a-97b2-b10cecbe31e2 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:30.559693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee9b07-2b7a-486a-97b2-b10cecbe31e2 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:30.559880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee9b07-2b7a-486a-97b2-b10cecbe31e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:30.593804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-bbee9b07-2b7a-486a-97b2-b10cecbe31e2 None None] MFA Rules not processed for user `e1daf1c8526245a9854f9b8672fa4222`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:30.595877 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-bbee9b07-2b7a-486a-97b2-b10cecbe31e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:30.692031 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 987/1930] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:30.702966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646d37e-a65e-4d63-a851-7ba328e41899 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:30.704449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b646d37e-a65e-4d63-a851-7ba328e41899 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:30.768592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646d37e-a65e-4d63-a851-7ba328e41899 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:30.772782 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28796162-6965-489b-a05d-e34a5326ee56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:30.773657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28796162-6965-489b-a05d-e34a5326ee56 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:30.773827 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28796162-6965-489b-a05d-e34a5326ee56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:30.774100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28796162-6965-489b-a05d-e34a5326ee56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:30.784933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28796162-6965-489b-a05d-e34a5326ee56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:30.786873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28796162-6965-489b-a05d-e34a5326ee56 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:30.796658 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 944/1931] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:09:32.364080 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448f2e2-ba8c-4a26-8c3b-b52395b70a12 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:32.364527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448f2e2-ba8c-4a26-8c3b-b52395b70a12 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:32.364793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448f2e2-ba8c-4a26-8c3b-b52395b70a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:32.383418 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-2448f2e2-ba8c-4a26-8c3b-b52395b70a12 None None] MFA Rules not processed for user `e1daf1c8526245a9854f9b8672fa4222`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:32.384685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-2448f2e2-ba8c-4a26-8c3b-b52395b70a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:32.392489 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 988/1932] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:40.791937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=62,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:42.377744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=56 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:49.915196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28796162-6965-489b-a05d-e34a5326ee56 service cinder] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:49.917440 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-28796162-6965-489b-a05d-e34a5326ee56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:49.958632 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28796162-6965-489b-a05d-e34a5326ee56 service cinder] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:49.978263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd333207-41c9-41f4-b22b-eb48b216c7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:49.979353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd333207-41c9-41f4-b22b-eb48b216c7cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:49.981169 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd333207-41c9-41f4-b22b-eb48b216c7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:49.981641 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd333207-41c9-41f4-b22b-eb48b216c7cf admin admin] PATH_INFO: `/v3/users/97ba745370674d858e93ae8ee5145730` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:49.983426 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd333207-41c9-41f4-b22b-eb48b216c7cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=97ba745370674d858e93ae8ee5145730)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:49.986216 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd333207-41c9-41f4-b22b-eb48b216c7cf admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:50.062824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-dd333207-41c9-41f4-b22b-eb48b216c7cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97ba745370674d858e93ae8ee5145730'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:50.064340 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-dd333207-41c9-41f4-b22b-eb48b216c7cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97ba745370674d858e93ae8ee5145730'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:50.085541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-dd333207-41c9-41f4-b22b-eb48b216c7cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97ba745370674d858e93ae8ee5145730'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:50.095007 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-dd333207-41c9-41f4-b22b-eb48b216c7cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97ba745370674d858e93ae8ee5145730'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:50.095798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-dd333207-41c9-41f4-b22b-eb48b216c7cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97ba745370674d858e93ae8ee5145730'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:50.118526 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 945/1933] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:09:49 2022] DELETE /identity/v3/users/97ba745370674d858e93ae8ee5145730 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:50.120620 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82265b83-ae1c-46ae-8311-056afd3d11b3 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:50.121179 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82265b83-ae1c-46ae-8311-056afd3d11b3 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:50.121638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82265b83-ae1c-46ae-8311-056afd3d11b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:50.147166 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-82265b83-ae1c-46ae-8311-056afd3d11b3 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:50.148461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-82265b83-ae1c-46ae-8311-056afd3d11b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:50.222738 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 989/1934] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:50.232538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd333207-41c9-41f4-b22b-eb48b216c7cf admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:50.234089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-dd333207-41c9-41f4-b22b-eb48b216c7cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:50.298845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd333207-41c9-41f4-b22b-eb48b216c7cf admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:50.302506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8b301-24e3-4f3e-801d-bb8c2abacfbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:50.303202 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8b301-24e3-4f3e-801d-bb8c2abacfbc service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:50.303591 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8b301-24e3-4f3e-801d-bb8c2abacfbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:50.303875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8b301-24e3-4f3e-801d-bb8c2abacfbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:50.313989 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f8b301-24e3-4f3e-801d-bb8c2abacfbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:50.315537 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f8b301-24e3-4f3e-801d-bb8c2abacfbc service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:50.325867 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 946/1935] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:09:50.748275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82265b83-ae1c-46ae-8311-056afd3d11b3 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:50.750608 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-82265b83-ae1c-46ae-8311-056afd3d11b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:50.766127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82265b83-ae1c-46ae-8311-056afd3d11b3 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:50.776939 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238368cf-2884-42b2-86c4-8caaa470afa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:50.777588 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238368cf-2884-42b2-86c4-8caaa470afa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:50.777863 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238368cf-2884-42b2-86c4-8caaa470afa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:50.778256 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238368cf-2884-42b2-86c4-8caaa470afa5 admin admin] PATH_INFO: `/v3/projects/90626527642f465990ec7b1f2fc9078d` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:50.779436 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238368cf-2884-42b2-86c4-8caaa470afa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90626527642f465990ec7b1f2fc9078d)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:50.782455 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238368cf-2884-42b2-86c4-8caaa470afa5 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:50.847793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-238368cf-2884-42b2-86c4-8caaa470afa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90626527642f465990ec7b1f2fc9078d', 'request_id': 'req-238368cf-2884-42b2-86c4-8caaa470afa5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:50.856221 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-238368cf-2884-42b2-86c4-8caaa470afa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90626527642f465990ec7b1f2fc9078d', 'request_id': 'req-238368cf-2884-42b2-86c4-8caaa470afa5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:50.862381 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-238368cf-2884-42b2-86c4-8caaa470afa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90626527642f465990ec7b1f2fc9078d', 'request_id': 'req-238368cf-2884-42b2-86c4-8caaa470afa5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:50.877434 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-238368cf-2884-42b2-86c4-8caaa470afa5 admin admin] The token cache is being invalidate because project 90626527642f465990ec7b1f2fc9078d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:09:50.877731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-238368cf-2884-42b2-86c4-8caaa470afa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:50.878424 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-238368cf-2884-42b2-86c4-8caaa470afa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:09:50.880424 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 990/1936] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:09:50 2022] DELETE /identity/v3/projects/90626527642f465990ec7b1f2fc9078d => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:50.885387 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceec225-0707-41d0-8bbf-ccf0209ec982 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:50.887136 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceec225-0707-41d0-8bbf-ccf0209ec982 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:50.887136 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceec225-0707-41d0-8bbf-ccf0209ec982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:50.908484 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-8ceec225-0707-41d0-8bbf-ccf0209ec982 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:50.911083 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-8ceec225-0707-41d0-8bbf-ccf0209ec982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:50.978063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 947/1937] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:09:50.986350 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62b3f7-1cb2-40f1-892e-4de4e9e9def9 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:50.986775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62b3f7-1cb2-40f1-892e-4de4e9e9def9 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:50.987215 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c62b3f7-1cb2-40f1-892e-4de4e9e9def9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:51.013986 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-3c62b3f7-1cb2-40f1-892e-4de4e9e9def9 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:51.015304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-3c62b3f7-1cb2-40f1-892e-4de4e9e9def9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:51.028468 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 991/1938] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:51.033398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceec225-0707-41d0-8bbf-ccf0209ec982 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:51.048311 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceec225-0707-41d0-8bbf-ccf0209ec982 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:51.053623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686b435-e4ca-4d27-98f8-18442c1eb80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:51.054478 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686b435-e4ca-4d27-98f8-18442c1eb80e admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:51.054759 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686b435-e4ca-4d27-98f8-18442c1eb80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:51.055032 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686b435-e4ca-4d27-98f8-18442c1eb80e admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:51.055741 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686b435-e4ca-4d27-98f8-18442c1eb80e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:09:51.056071 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686b435-e4ca-4d27-98f8-18442c1eb80e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:51.057662 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686b435-e4ca-4d27-98f8-18442c1eb80e admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:51.076707 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 948/1939] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:09:51.081136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62b3f7-1cb2-40f1-892e-4de4e9e9def9 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:51.091381 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c62b3f7-1cb2-40f1-892e-4de4e9e9def9 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:51.096615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd1aea-b535-4c20-9447-4fe70e8a7ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:51.097210 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd1aea-b535-4c20-9447-4fe70e8a7ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:51.097488 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd1aea-b535-4c20-9447-4fe70e8a7ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:51.097755 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd1aea-b535-4c20-9447-4fe70e8a7ee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:51.098388 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd1aea-b535-4c20-9447-4fe70e8a7ee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:51.100301 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd1aea-b535-4c20-9447-4fe70e8a7ee3 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:51.127186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 992/1940] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:09:51 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:09:51.131302 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686b435-e4ca-4d27-98f8-18442c1eb80e admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:51.136409 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:51.142957 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686b435-e4ca-4d27-98f8-18442c1eb80e admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:51.216333 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a349f-3f71-4de4-8f20-9eb5b4264ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:51.216954 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a349f-3f71-4de4-8f20-9eb5b4264ce2 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:51.217235 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a349f-3f71-4de4-8f20-9eb5b4264ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:51.217610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a349f-3f71-4de4-8f20-9eb5b4264ce2 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:51.219190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a349f-3f71-4de4-8f20-9eb5b4264ce2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:51.221710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467a349f-3f71-4de4-8f20-9eb5b4264ce2 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:51.243468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 949/1941] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:09:51 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:09:51.247822 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd1aea-b535-4c20-9447-4fe70e8a7ee3 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:51.258072 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd1aea-b535-4c20-9447-4fe70e8a7ee3 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:51.262632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e48eae-5553-4f43-827e-832e3c639488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:51.263373 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e48eae-5553-4f43-827e-832e3c639488 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:51.263655 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e48eae-5553-4f43-827e-832e3c639488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:51.263925 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e48eae-5553-4f43-827e-832e3c639488 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:51.264558 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e48eae-5553-4f43-827e-832e3c639488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:51.266158 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e48eae-5553-4f43-827e-832e3c639488 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:51.276652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 993/1942] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:09:51.280715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a349f-3f71-4de4-8f20-9eb5b4264ce2 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:51.292442 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a349f-3f71-4de4-8f20-9eb5b4264ce2 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:51.297317 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3166aa-4ee6-4c17-b246-d3b75a9767ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:51.297950 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3166aa-4ee6-4c17-b246-d3b75a9767ad admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:51.298231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3166aa-4ee6-4c17-b246-d3b75a9767ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:51.298505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3166aa-4ee6-4c17-b246-d3b75a9767ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:51.299215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3166aa-4ee6-4c17-b246-d3b75a9767ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:51.300841 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3166aa-4ee6-4c17-b246-d3b75a9767ad admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:51.317693 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 950/1943] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:09:51.321982 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e48eae-5553-4f43-827e-832e3c639488 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:51.339720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e48eae-5553-4f43-827e-832e3c639488 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:51.346623 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5017581-3f54-4398-a58d-f3cff62693ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:51.347698 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5017581-3f54-4398-a58d-f3cff62693ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:51.348153 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5017581-3f54-4398-a58d-f3cff62693ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:51.348666 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5017581-3f54-4398-a58d-f3cff62693ec admin admin] PATH_INFO: `/v3/projects/d1883a19a6d24b36a063a5e673495a51/users/66133313929d4c0f98ab21b1ea336456/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:51.369958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5017581-3f54-4398-a58d-f3cff62693ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1883a19a6d24b36a063a5e673495a51, user_id=66133313929d4c0f98ab21b1ea336456, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:51.372776 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5017581-3f54-4398-a58d-f3cff62693ec admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:51.388345 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 994/1944] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:09:51 2022] PUT /identity/v3/projects/d1883a19a6d24b36a063a5e673495a51/users/66133313929d4c0f98ab21b1ea336456/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:09:51.394655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da927b-8312-4ae5-8f64-dc22a581218a None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:51.394655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da927b-8312-4ae5-8f64-dc22a581218a None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:51.394655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da927b-8312-4ae5-8f64-dc22a581218a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:51.421846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-a3da927b-8312-4ae5-8f64-dc22a581218a None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:51.423246 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-a3da927b-8312-4ae5-8f64-dc22a581218a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:51.491106 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 951/1945] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:51.498225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5017581-3f54-4398-a58d-f3cff62693ec admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:51.500217 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c5017581-3f54-4398-a58d-f3cff62693ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:51.527297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:09:51.564628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5017581-3f54-4398-a58d-f3cff62693ec admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:51.568989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7d54d-3d3b-4cf4-a8c5-908f0b7447d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:51.569748 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7d54d-3d3b-4cf4-a8c5-908f0b7447d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:51.570247 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7d54d-3d3b-4cf4-a8c5-908f0b7447d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:51.570553 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec7d54d-3d3b-4cf4-a8c5-908f0b7447d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:51.581610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7d54d-3d3b-4cf4-a8c5-908f0b7447d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:51.583055 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec7d54d-3d3b-4cf4-a8c5-908f0b7447d2 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:51.592872 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 995/1946] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:09:52.998079 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43382582-8dae-40be-8ad1-00f643d5098e None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:52.998645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43382582-8dae-40be-8ad1-00f643d5098e None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:52.998901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43382582-8dae-40be-8ad1-00f643d5098e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:53.031213 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-43382582-8dae-40be-8ad1-00f643d5098e None None] MFA Rules not processed for user `66133313929d4c0f98ab21b1ea336456`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:53.032429 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-43382582-8dae-40be-8ad1-00f643d5098e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:53.132239 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7d54d-3d3b-4cf4-a8c5-908f0b7447d2 service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:09:53.133443 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 952/1947] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:09:53.134549 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-1ec7d54d-3d3b-4cf4-a8c5-908f0b7447d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:53.196155 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec7d54d-3d3b-4cf4-a8c5-908f0b7447d2 service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:09:53.200177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3dc472-e9e3-4240-9026-055484d273e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:09:53.200786 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3dc472-e9e3-4240-9026-055484d273e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:53.201250 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3dc472-e9e3-4240-9026-055484d273e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:53.201526 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3dc472-e9e3-4240-9026-055484d273e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:53.211500 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3dc472-e9e3-4240-9026-055484d273e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:09:53.212941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3dc472-e9e3-4240-9026-055484d273e1 service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:09:53.222017 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 996/1948] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:09:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:09:54.680199 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc19bc-2879-4d8f-918f-3a49200141e3 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:09:54.680663 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc19bc-2879-4d8f-918f-3a49200141e3 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:09:54.680751 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc19bc-2879-4d8f-918f-3a49200141e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:09:54.700861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-89cc19bc-2879-4d8f-918f-3a49200141e3 None None] MFA Rules not processed for user `66133313929d4c0f98ab21b1ea336456`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:09:54.702157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-89cc19bc-2879-4d8f-918f-3a49200141e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:09:54.710098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 953/1949] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:09:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:01.104776 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3dc472-e9e3-4240-9026-055484d273e1 service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:01.106202 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-7b3dc472-e9e3-4240-9026-055484d273e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:01.119609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3dc472-e9e3-4240-9026-055484d273e1 service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:01.124083 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9334738-9671-42a4-b7e0-95169707c218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:01.124686 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9334738-9671-42a4-b7e0-95169707c218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:01.124964 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9334738-9671-42a4-b7e0-95169707c218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:01.125233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9334738-9671-42a4-b7e0-95169707c218 admin admin] PATH_INFO: `/v3/users/5b755e93453c4c2594afe759f6c4bf0f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:01.126341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9334738-9671-42a4-b7e0-95169707c218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b755e93453c4c2594afe759f6c4bf0f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:01.128558 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9334738-9671-42a4-b7e0-95169707c218 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:01.181102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-a9334738-9671-42a4-b7e0-95169707c218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b755e93453c4c2594afe759f6c4bf0f'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:01.188176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-a9334738-9671-42a4-b7e0-95169707c218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b755e93453c4c2594afe759f6c4bf0f'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:01.196441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-a9334738-9671-42a4-b7e0-95169707c218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b755e93453c4c2594afe759f6c4bf0f'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:01.197022 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-a9334738-9671-42a4-b7e0-95169707c218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b755e93453c4c2594afe759f6c4bf0f'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:01.197859 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-a9334738-9671-42a4-b7e0-95169707c218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b755e93453c4c2594afe759f6c4bf0f'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:01.215999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 997/1950] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:10:01 2022] DELETE /identity/v3/users/5b755e93453c4c2594afe759f6c4bf0f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:01.220623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b450b1-f56d-4a8c-9043-c9e067ddf433 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:01.221088 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b450b1-f56d-4a8c-9043-c9e067ddf433 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:01.221354 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b450b1-f56d-4a8c-9043-c9e067ddf433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:01.229848 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:01.241725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-48b450b1-f56d-4a8c-9043-c9e067ddf433 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:01.243024 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-48b450b1-f56d-4a8c-9043-c9e067ddf433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:01.301412 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 954/1951] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:10:01.311021 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9334738-9671-42a4-b7e0-95169707c218 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:01.312530 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a9334738-9671-42a4-b7e0-95169707c218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:01.357517 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:01.373805 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9334738-9671-42a4-b7e0-95169707c218 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:01.377987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efc8ff-9117-4a72-861b-54b58b1f2e80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:01.378584 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efc8ff-9117-4a72-861b-54b58b1f2e80 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:01.378858 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efc8ff-9117-4a72-861b-54b58b1f2e80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:01.379158 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efc8ff-9117-4a72-861b-54b58b1f2e80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:01.388067 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66efc8ff-9117-4a72-861b-54b58b1f2e80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:01.389485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66efc8ff-9117-4a72-861b-54b58b1f2e80 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:01.399539 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 998/1952] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:10:01.772283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b450b1-f56d-4a8c-9043-c9e067ddf433 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:01.773877 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-48b450b1-f56d-4a8c-9043-c9e067ddf433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:01.792025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b450b1-f56d-4a8c-9043-c9e067ddf433 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:01.798038 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe67954-ecaf-4d11-97ea-da8480f418e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:01.798746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe67954-ecaf-4d11-97ea-da8480f418e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:01.799113 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe67954-ecaf-4d11-97ea-da8480f418e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:01.799455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe67954-ecaf-4d11-97ea-da8480f418e1 admin admin] PATH_INFO: `/v3/projects/34750dac24ce4283a8f9f8697773bbe7` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:01.800722 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe67954-ecaf-4d11-97ea-da8480f418e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34750dac24ce4283a8f9f8697773bbe7)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:01.802663 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe67954-ecaf-4d11-97ea-da8480f418e1 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:01.853311 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9fe67954-ecaf-4d11-97ea-da8480f418e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34750dac24ce4283a8f9f8697773bbe7', 'request_id': 'req-9fe67954-ecaf-4d11-97ea-da8480f418e1'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:01.859326 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9fe67954-ecaf-4d11-97ea-da8480f418e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34750dac24ce4283a8f9f8697773bbe7', 'request_id': 'req-9fe67954-ecaf-4d11-97ea-da8480f418e1'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:01.865228 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9fe67954-ecaf-4d11-97ea-da8480f418e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34750dac24ce4283a8f9f8697773bbe7', 'request_id': 'req-9fe67954-ecaf-4d11-97ea-da8480f418e1'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:01.877891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9fe67954-ecaf-4d11-97ea-da8480f418e1 admin admin] The token cache is being invalidate because project 34750dac24ce4283a8f9f8697773bbe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:10:01.878183 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9fe67954-ecaf-4d11-97ea-da8480f418e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:01.878809 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9fe67954-ecaf-4d11-97ea-da8480f418e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:01.880319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 955/1953] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:10:01 2022] DELETE /identity/v3/projects/34750dac24ce4283a8f9f8697773bbe7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:01.884905 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efc8ff-9117-4a72-861b-54b58b1f2e80 service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:01.894476 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efc8ff-9117-4a72-861b-54b58b1f2e80 service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:01.950549 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa94bd8-f7cb-410e-b373-255e775548ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:01.951356 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa94bd8-f7cb-410e-b373-255e775548ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:01.951537 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa94bd8-f7cb-410e-b373-255e775548ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:01.951804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa94bd8-f7cb-410e-b373-255e775548ea admin admin] PATH_INFO: `/v3/users/920108cf870b4865843083e0c3fd2aed` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:01.952907 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa94bd8-f7cb-410e-b373-255e775548ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=920108cf870b4865843083e0c3fd2aed)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:01.954788 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa94bd8-f7cb-410e-b373-255e775548ea admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:02.001438 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-8aa94bd8-f7cb-410e-b373-255e775548ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '920108cf870b4865843083e0c3fd2aed'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:02.009290 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-8aa94bd8-f7cb-410e-b373-255e775548ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '920108cf870b4865843083e0c3fd2aed'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:02.016918 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-8aa94bd8-f7cb-410e-b373-255e775548ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '920108cf870b4865843083e0c3fd2aed'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:02.017692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-8aa94bd8-f7cb-410e-b373-255e775548ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '920108cf870b4865843083e0c3fd2aed'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:02.018380 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-8aa94bd8-f7cb-410e-b373-255e775548ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '920108cf870b4865843083e0c3fd2aed'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:02.037842 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 999/1954] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:10:01 2022] DELETE /identity/v3/users/920108cf870b4865843083e0c3fd2aed => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:02.403035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe67954-ecaf-4d11-97ea-da8480f418e1 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:02.404554 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-9fe67954-ecaf-4d11-97ea-da8480f418e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:02.418433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe67954-ecaf-4d11-97ea-da8480f418e1 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:02.486045 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9b723-cfd8-45bf-a96e-bde5e64a8a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:02.487018 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9b723-cfd8-45bf-a96e-bde5e64a8a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:02.487335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9b723-cfd8-45bf-a96e-bde5e64a8a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:02.487787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9b723-cfd8-45bf-a96e-bde5e64a8a29 admin admin] PATH_INFO: `/v3/projects/a987b62a9a174e3f9da5b19cc69d86de` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:02.488956 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9b723-cfd8-45bf-a96e-bde5e64a8a29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a987b62a9a174e3f9da5b19cc69d86de)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:02.490884 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9b723-cfd8-45bf-a96e-bde5e64a8a29 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:02.525724 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:02.540063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b3a9b723-cfd8-45bf-a96e-bde5e64a8a29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a987b62a9a174e3f9da5b19cc69d86de', 'request_id': 'req-b3a9b723-cfd8-45bf-a96e-bde5e64a8a29'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:02.546253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b3a9b723-cfd8-45bf-a96e-bde5e64a8a29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a987b62a9a174e3f9da5b19cc69d86de', 'request_id': 'req-b3a9b723-cfd8-45bf-a96e-bde5e64a8a29'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:02.552134 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b3a9b723-cfd8-45bf-a96e-bde5e64a8a29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a987b62a9a174e3f9da5b19cc69d86de', 'request_id': 'req-b3a9b723-cfd8-45bf-a96e-bde5e64a8a29'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:02.565849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b3a9b723-cfd8-45bf-a96e-bde5e64a8a29 admin admin] The token cache is being invalidate because project a987b62a9a174e3f9da5b19cc69d86de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:10:02.566240 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b3a9b723-cfd8-45bf-a96e-bde5e64a8a29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:02.567049 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b3a9b723-cfd8-45bf-a96e-bde5e64a8a29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:02.568778 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 956/1955] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:10:02 2022] DELETE /identity/v3/projects/a987b62a9a174e3f9da5b19cc69d86de => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:10:02.573827 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8425c-5691-4b74-9ad0-a478946b2333 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:02.574281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8425c-5691-4b74-9ad0-a478946b2333 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:02.574411 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8425c-5691-4b74-9ad0-a478946b2333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:02.594004 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-14a8425c-5691-4b74-9ad0-a478946b2333 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:02.595348 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-14a8425c-5691-4b74-9ad0-a478946b2333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:02.648405 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:02.658801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1000/1956] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:02.666110 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761868bc-21dc-47c4-a95c-db96ce54bd29 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:02.666394 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761868bc-21dc-47c4-a95c-db96ce54bd29 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:02.666668 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761868bc-21dc-47c4-a95c-db96ce54bd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:02.687415 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-761868bc-21dc-47c4-a95c-db96ce54bd29 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:02.689281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-761868bc-21dc-47c4-a95c-db96ce54bd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:02.697224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 957/1957] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:02.701876 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8425c-5691-4b74-9ad0-a478946b2333 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:02.711696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8425c-5691-4b74-9ad0-a478946b2333 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:02.716765 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132b03c-6d38-441d-a810-c7ca6fe11ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:02.717819 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132b03c-6d38-441d-a810-c7ca6fe11ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:02.718108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132b03c-6d38-441d-a810-c7ca6fe11ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:02.718981 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132b03c-6d38-441d-a810-c7ca6fe11ff3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:02.719263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6132b03c-6d38-441d-a810-c7ca6fe11ff3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:10:02.719701 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6132b03c-6d38-441d-a810-c7ca6fe11ff3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:02.721416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6132b03c-6d38-441d-a810-c7ca6fe11ff3 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:02.734222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1001/1958] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:10:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:10:02.738880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761868bc-21dc-47c4-a95c-db96ce54bd29 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:02.748136 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761868bc-21dc-47c4-a95c-db96ce54bd29 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:02.753101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84417a-f0f5-4739-8eaf-cf32d94e36f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:02.753739 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84417a-f0f5-4739-8eaf-cf32d94e36f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:02.754017 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84417a-f0f5-4739-8eaf-cf32d94e36f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:02.754306 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84417a-f0f5-4739-8eaf-cf32d94e36f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:02.754967 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84417a-f0f5-4739-8eaf-cf32d94e36f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:02.756897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84417a-f0f5-4739-8eaf-cf32d94e36f8 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:02.789403 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 958/1959] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:10:02 2022] POST /identity/v3/projects => generated 357 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:10:02.794501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132b03c-6d38-441d-a810-c7ca6fe11ff3 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:02.805069 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132b03c-6d38-441d-a810-c7ca6fe11ff3 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:02.867493 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc8da5-638e-47b7-9cff-a0fdc31ce225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:02.868132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc8da5-638e-47b7-9cff-a0fdc31ce225 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:02.868427 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc8da5-638e-47b7-9cff-a0fdc31ce225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:02.868727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc8da5-638e-47b7-9cff-a0fdc31ce225 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:02.869544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc8da5-638e-47b7-9cff-a0fdc31ce225 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:02.871570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc8da5-638e-47b7-9cff-a0fdc31ce225 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:02.894566 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1002/1960] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:10:02 2022] POST /identity/v3/users => generated 279 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:02.898789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84417a-f0f5-4739-8eaf-cf32d94e36f8 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:02.910294 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84417a-f0f5-4739-8eaf-cf32d94e36f8 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:02.918117 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac8fc9-1006-4cdb-99d5-aa71577953e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:02.918117 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac8fc9-1006-4cdb-99d5-aa71577953e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:02.918117 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac8fc9-1006-4cdb-99d5-aa71577953e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:02.918117 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac8fc9-1006-4cdb-99d5-aa71577953e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:02.918473 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac8fc9-1006-4cdb-99d5-aa71577953e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:02.918563 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac8fc9-1006-4cdb-99d5-aa71577953e9 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:02.929439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 959/1961] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:02.933533 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc8da5-638e-47b7-9cff-a0fdc31ce225 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:02.943875 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc8da5-638e-47b7-9cff-a0fdc31ce225 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:02.948906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9041ab1-ff8e-4e44-9268-431e5b76e466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:02.949526 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9041ab1-ff8e-4e44-9268-431e5b76e466 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:02.949810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9041ab1-ff8e-4e44-9268-431e5b76e466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:02.950084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9041ab1-ff8e-4e44-9268-431e5b76e466 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:02.950724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9041ab1-ff8e-4e44-9268-431e5b76e466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:02.952363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9041ab1-ff8e-4e44-9268-431e5b76e466 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:02.962836 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1003/1962] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:02.966886 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac8fc9-1006-4cdb-99d5-aa71577953e9 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:02.977452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac8fc9-1006-4cdb-99d5-aa71577953e9 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:02.982026 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8244c-42a8-449b-82ac-9510c19138ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:02.982668 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8244c-42a8-449b-82ac-9510c19138ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:02.982951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8244c-42a8-449b-82ac-9510c19138ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:02.983257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8244c-42a8-449b-82ac-9510c19138ab admin admin] PATH_INFO: `/v3/projects/0ae24aa029a1481087bd9e0387035272/users/6080ff93f4144848b63e9a177f85fdf9/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:02.998400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8244c-42a8-449b-82ac-9510c19138ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ae24aa029a1481087bd9e0387035272, user_id=6080ff93f4144848b63e9a177f85fdf9, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:03.001400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8244c-42a8-449b-82ac-9510c19138ab admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:03.014989 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 960/1963] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:02 2022] PUT /identity/v3/projects/0ae24aa029a1481087bd9e0387035272/users/6080ff93f4144848b63e9a177f85fdf9/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:03.025213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a957db-a8e8-49a2-8c2f-bc4bbdb90c29 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:03.025499 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a957db-a8e8-49a2-8c2f-bc4bbdb90c29 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:03.025769 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a957db-a8e8-49a2-8c2f-bc4bbdb90c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:03.054333 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-c1a957db-a8e8-49a2-8c2f-bc4bbdb90c29 None None] MFA Rules not processed for user `6080ff93f4144848b63e9a177f85fdf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:03.055629 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c1a957db-a8e8-49a2-8c2f-bc4bbdb90c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:03.124420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1004/1964] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:03 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:03.128855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8244c-42a8-449b-82ac-9510c19138ab admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:03.196279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8244c-42a8-449b-82ac-9510c19138ab admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:03.201150 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e636d5-6db9-455e-bbb8-0dd85ba91682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:03.201760 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e636d5-6db9-455e-bbb8-0dd85ba91682 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:03.202042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e636d5-6db9-455e-bbb8-0dd85ba91682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:03.202313 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e636d5-6db9-455e-bbb8-0dd85ba91682 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:03.202946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e636d5-6db9-455e-bbb8-0dd85ba91682 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:03.205413 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e636d5-6db9-455e-bbb8-0dd85ba91682 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:03.236988 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 961/1965] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:10:03 2022] POST /identity/v3/projects => generated 357 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:03.241033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a957db-a8e8-49a2-8c2f-bc4bbdb90c29 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:03.255966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a957db-a8e8-49a2-8c2f-bc4bbdb90c29 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:03.316795 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:03.327293 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d6870-1c2c-46a2-9316-c32203e53e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:03.328077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d6870-1c2c-46a2-9316-c32203e53e93 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:03.328630 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d6870-1c2c-46a2-9316-c32203e53e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:03.329125 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d6870-1c2c-46a2-9316-c32203e53e93 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:03.330153 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d6870-1c2c-46a2-9316-c32203e53e93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:03.332406 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d6870-1c2c-46a2-9316-c32203e53e93 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:03.355375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1005/1966] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:10:03 2022] POST /identity/v3/users => generated 285 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:10:03.359339 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e636d5-6db9-455e-bbb8-0dd85ba91682 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:03.370042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e636d5-6db9-455e-bbb8-0dd85ba91682 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:03.374740 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083d8a5-db49-4ceb-8621-24ccfb1b9cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:03.375307 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083d8a5-db49-4ceb-8621-24ccfb1b9cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:03.375588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083d8a5-db49-4ceb-8621-24ccfb1b9cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:03.375922 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083d8a5-db49-4ceb-8621-24ccfb1b9cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:03.376598 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083d8a5-db49-4ceb-8621-24ccfb1b9cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:03.378183 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083d8a5-db49-4ceb-8621-24ccfb1b9cf1 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:03.388501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 962/1967] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:03.393139 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d6870-1c2c-46a2-9316-c32203e53e93 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:03.403181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d6870-1c2c-46a2-9316-c32203e53e93 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:03.408507 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7777bc0-7b3c-4c62-9663-1867143df5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:03.409165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7777bc0-7b3c-4c62-9663-1867143df5c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:03.409445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7777bc0-7b3c-4c62-9663-1867143df5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:03.409725 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7777bc0-7b3c-4c62-9663-1867143df5c0 admin admin] PATH_INFO: `/v3/projects/e41533102c2d49d38c3337a960e8bbaf/users/55e167f8d9224fe698fa9c8a05e947ec/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:03.425291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7777bc0-7b3c-4c62-9663-1867143df5c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e41533102c2d49d38c3337a960e8bbaf, user_id=55e167f8d9224fe698fa9c8a05e947ec, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:03.428157 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7777bc0-7b3c-4c62-9663-1867143df5c0 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:03.443168 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1006/1968] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:03 2022] PUT /identity/v3/projects/e41533102c2d49d38c3337a960e8bbaf/users/55e167f8d9224fe698fa9c8a05e947ec/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:03.447088 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083d8a5-db49-4ceb-8621-24ccfb1b9cf1 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:03.458193 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083d8a5-db49-4ceb-8621-24ccfb1b9cf1 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:03.484179 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:03.517063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa3c84-9024-4165-b3b3-bc15bb7687d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:03.517673 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa3c84-9024-4165-b3b3-bc15bb7687d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:03.518450 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa3c84-9024-4165-b3b3-bc15bb7687d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:03.518746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa3c84-9024-4165-b3b3-bc15bb7687d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:03.519414 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa3c84-9024-4165-b3b3-bc15bb7687d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:03.520999 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa3c84-9024-4165-b3b3-bc15bb7687d5 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:03.530668 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 963/1969] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:03.534795 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7777bc0-7b3c-4c62-9663-1867143df5c0 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:03.545037 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7777bc0-7b3c-4c62-9663-1867143df5c0 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:03.549857 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7171e-c419-4114-a0ce-4dfb348407ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:03.550511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7171e-c419-4114-a0ce-4dfb348407ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:03.550791 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7171e-c419-4114-a0ce-4dfb348407ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:03.551129 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7171e-c419-4114-a0ce-4dfb348407ff admin admin] PATH_INFO: `/v3/projects/e41533102c2d49d38c3337a960e8bbaf/users/55e167f8d9224fe698fa9c8a05e947ec/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:03.553053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7171e-c419-4114-a0ce-4dfb348407ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=e41533102c2d49d38c3337a960e8bbaf, user_id=55e167f8d9224fe698fa9c8a05e947ec, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:03.555755 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b7171e-c419-4114-a0ce-4dfb348407ff admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:03.566323 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1007/1970] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:03 2022] PUT /identity/v3/projects/e41533102c2d49d38c3337a960e8bbaf/users/55e167f8d9224fe698fa9c8a05e947ec/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:03.577505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb38ec-22b1-45f4-b665-9e44611b19bc None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:03.577895 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb38ec-22b1-45f4-b665-9e44611b19bc None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:03.578173 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb38ec-22b1-45f4-b665-9e44611b19bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:03.606233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-aceb38ec-22b1-45f4-b665-9e44611b19bc None None] MFA Rules not processed for user `55e167f8d9224fe698fa9c8a05e947ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:03.607477 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-aceb38ec-22b1-45f4-b665-9e44611b19bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:03.677418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 964/1971] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:03 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:10:03.688558 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7171e-c419-4114-a0ce-4dfb348407ff admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:03.690369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-36b7171e-c419-4114-a0ce-4dfb348407ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:03.747927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7171e-c419-4114-a0ce-4dfb348407ff admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:03.751597 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d886d-3d56-4a9a-8c9b-db645dea2c45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:03.752188 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d886d-3d56-4a9a-8c9b-db645dea2c45 service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:03.752462 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d886d-3d56-4a9a-8c9b-db645dea2c45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:03.752729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d886d-3d56-4a9a-8c9b-db645dea2c45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:03.761486 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d886d-3d56-4a9a-8c9b-db645dea2c45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:03.762896 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d886d-3d56-4a9a-8c9b-db645dea2c45 service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:03.772347 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1008/1972] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:03 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:10:03.853443 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb38ec-22b1-45f4-b665-9e44611b19bc None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:03.867193 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb38ec-22b1-45f4-b665-9e44611b19bc None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:03.880375 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb2afb-0399-49d8-aa77-241ad468f6d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:03.882875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb2afb-0399-49d8-aa77-241ad468f6d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:03.883215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb2afb-0399-49d8-aa77-241ad468f6d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:03.883668 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb2afb-0399-49d8-aa77-241ad468f6d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:03.959361 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb2afb-0399-49d8-aa77-241ad468f6d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:03.960797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb2afb-0399-49d8-aa77-241ad468f6d1 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:03.970432 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 965/1973] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:03 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:10:13.767944 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=45,keystone:INSERT=5 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:13.966177 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=78 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:14.283880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d886d-3d56-4a9a-8c9b-db645dea2c45 service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:14.285374 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-eb6d886d-3d56-4a9a-8c9b-db645dea2c45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:14.351030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d886d-3d56-4a9a-8c9b-db645dea2c45 service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:14.355561 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cbb451-af8c-4575-b31b-5e6ffc7be17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:14.356233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbb451-af8c-4575-b31b-5e6ffc7be17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:14.356509 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbb451-af8c-4575-b31b-5e6ffc7be17c admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:14.356773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbb451-af8c-4575-b31b-5e6ffc7be17c admin admin] PATH_INFO: `/v3/users/3adaaf8d979d4cf28456812f9dcfa809` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:14.357882 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cbb451-af8c-4575-b31b-5e6ffc7be17c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3adaaf8d979d4cf28456812f9dcfa809)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:14.359812 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cbb451-af8c-4575-b31b-5e6ffc7be17c admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:14.408389 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-30cbb451-af8c-4575-b31b-5e6ffc7be17c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3adaaf8d979d4cf28456812f9dcfa809'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:14.415152 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-30cbb451-af8c-4575-b31b-5e6ffc7be17c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3adaaf8d979d4cf28456812f9dcfa809'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:14.422175 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-30cbb451-af8c-4575-b31b-5e6ffc7be17c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3adaaf8d979d4cf28456812f9dcfa809'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:14.422796 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-30cbb451-af8c-4575-b31b-5e6ffc7be17c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3adaaf8d979d4cf28456812f9dcfa809'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:14.423464 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-30cbb451-af8c-4575-b31b-5e6ffc7be17c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3adaaf8d979d4cf28456812f9dcfa809'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:14.440121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1009/1974] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:10:14 2022] DELETE /identity/v3/users/3adaaf8d979d4cf28456812f9dcfa809 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:14.613816 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb2afb-0399-49d8-aa77-241ad468f6d1 service cinder] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:14.634792 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb2afb-0399-49d8-aa77-241ad468f6d1 service cinder] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:14.700587 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49590483-54f1-417a-9ff9-b401a00d1739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:14.701218 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49590483-54f1-417a-9ff9-b401a00d1739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:14.701491 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49590483-54f1-417a-9ff9-b401a00d1739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:14.701776 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49590483-54f1-417a-9ff9-b401a00d1739 admin admin] PATH_INFO: `/v3/projects/8ebe6b70a4374511b388d20265e4ccdf` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:14.702865 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49590483-54f1-417a-9ff9-b401a00d1739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ebe6b70a4374511b388d20265e4ccdf)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:14.705371 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49590483-54f1-417a-9ff9-b401a00d1739 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:14.750187 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-49590483-54f1-417a-9ff9-b401a00d1739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ebe6b70a4374511b388d20265e4ccdf', 'request_id': 'req-49590483-54f1-417a-9ff9-b401a00d1739'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:14.756645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-49590483-54f1-417a-9ff9-b401a00d1739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ebe6b70a4374511b388d20265e4ccdf', 'request_id': 'req-49590483-54f1-417a-9ff9-b401a00d1739'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:14.762815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-49590483-54f1-417a-9ff9-b401a00d1739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ebe6b70a4374511b388d20265e4ccdf', 'request_id': 'req-49590483-54f1-417a-9ff9-b401a00d1739'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:14.777675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-49590483-54f1-417a-9ff9-b401a00d1739 admin admin] The token cache is being invalidate because project 8ebe6b70a4374511b388d20265e4ccdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:10:14.777965 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-49590483-54f1-417a-9ff9-b401a00d1739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:14.778610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-49590483-54f1-417a-9ff9-b401a00d1739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:14.780229 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 966/1975] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:10:14 2022] DELETE /identity/v3/projects/8ebe6b70a4374511b388d20265e4ccdf => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:14.788326 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e9f46-1860-4cbf-bd2d-e114f06baad9 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:14.788867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e9f46-1860-4cbf-bd2d-e114f06baad9 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:14.788867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441e9f46-1860-4cbf-bd2d-e114f06baad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:14.810281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-441e9f46-1860-4cbf-bd2d-e114f06baad9 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:14.811506 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-441e9f46-1860-4cbf-bd2d-e114f06baad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:14.869053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1010/1976] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:14.877326 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56afb9c-5f1b-4561-a4b3-3ae91aaa3698 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:14.877614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56afb9c-5f1b-4561-a4b3-3ae91aaa3698 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:14.877890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56afb9c-5f1b-4561-a4b3-3ae91aaa3698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:14.900386 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-d56afb9c-5f1b-4561-a4b3-3ae91aaa3698 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:14.901866 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-d56afb9c-5f1b-4561-a4b3-3ae91aaa3698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:14.912210 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 967/1977] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:14.916349 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441e9f46-1860-4cbf-bd2d-e114f06baad9 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:14.927351 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441e9f46-1860-4cbf-bd2d-e114f06baad9 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:14.932450 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c5bf8-429f-4fc5-9c1f-f3d3c04feaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:14.933336 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c5bf8-429f-4fc5-9c1f-f3d3c04feaa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:14.933617 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c5bf8-429f-4fc5-9c1f-f3d3c04feaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:14.933891 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68c5bf8-429f-4fc5-9c1f-f3d3c04feaa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:14.934542 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c5bf8-429f-4fc5-9c1f-f3d3c04feaa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:10:14.934847 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c5bf8-429f-4fc5-9c1f-f3d3c04feaa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:14.936498 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68c5bf8-429f-4fc5-9c1f-f3d3c04feaa8 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:14.959318 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1011/1978] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:10:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:10:14.960759 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56afb9c-5f1b-4561-a4b3-3ae91aaa3698 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:14.970418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56afb9c-5f1b-4561-a4b3-3ae91aaa3698 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:14.975215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16a7f8-5ca6-415e-b70d-0ca004bc3386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:14.975926 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16a7f8-5ca6-415e-b70d-0ca004bc3386 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:14.976275 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16a7f8-5ca6-415e-b70d-0ca004bc3386 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:14.976619 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16a7f8-5ca6-415e-b70d-0ca004bc3386 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:14.977327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16a7f8-5ca6-415e-b70d-0ca004bc3386 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:14.979360 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16a7f8-5ca6-415e-b70d-0ca004bc3386 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:15.012936 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 968/1979] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:10:14 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:15.014907 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c5bf8-429f-4fc5-9c1f-f3d3c04feaa8 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:15.026672 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68c5bf8-429f-4fc5-9c1f-f3d3c04feaa8 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:15.030499 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:15.098000 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12043942-cd48-4e49-a101-741cc06cf522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:15.098649 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12043942-cd48-4e49-a101-741cc06cf522 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:15.099009 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12043942-cd48-4e49-a101-741cc06cf522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:15.099374 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12043942-cd48-4e49-a101-741cc06cf522 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:15.100087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12043942-cd48-4e49-a101-741cc06cf522 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:15.102017 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12043942-cd48-4e49-a101-741cc06cf522 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:15.124629 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1012/1980] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:10:15 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:15.129021 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16a7f8-5ca6-415e-b70d-0ca004bc3386 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:15.157203 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16a7f8-5ca6-415e-b70d-0ca004bc3386 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:15.163367 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622378bc-f794-445a-8a87-964997e61fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:15.164139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622378bc-f794-445a-8a87-964997e61fac admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:15.164560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622378bc-f794-445a-8a87-964997e61fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:15.165043 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622378bc-f794-445a-8a87-964997e61fac admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:15.165948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622378bc-f794-445a-8a87-964997e61fac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:15.167980 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622378bc-f794-445a-8a87-964997e61fac admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:15.187248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 969/1981] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:15.191384 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12043942-cd48-4e49-a101-741cc06cf522 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:15.204932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12043942-cd48-4e49-a101-741cc06cf522 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:15.210490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0275af-d6bf-479e-9c8a-2f1b9756fad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:15.211115 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0275af-d6bf-479e-9c8a-2f1b9756fad3 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:15.211397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0275af-d6bf-479e-9c8a-2f1b9756fad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:15.211675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0275af-d6bf-479e-9c8a-2f1b9756fad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:15.212309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0275af-d6bf-479e-9c8a-2f1b9756fad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:15.213914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0275af-d6bf-479e-9c8a-2f1b9756fad3 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:15.226635 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1013/1982] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:15.231092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622378bc-f794-445a-8a87-964997e61fac admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:15.242586 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622378bc-f794-445a-8a87-964997e61fac admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:15.248028 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a38c44-5bac-481a-9119-dc0fc70736ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:15.248840 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a38c44-5bac-481a-9119-dc0fc70736ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:15.249414 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a38c44-5bac-481a-9119-dc0fc70736ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:15.250132 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a38c44-5bac-481a-9119-dc0fc70736ec admin admin] PATH_INFO: `/v3/projects/5e37db690681428f8e47bc6f46b7dbd7/users/c49c8104d02f4431a94f165206717531/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:15.268752 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a38c44-5bac-481a-9119-dc0fc70736ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e37db690681428f8e47bc6f46b7dbd7, user_id=c49c8104d02f4431a94f165206717531, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:15.271746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a38c44-5bac-481a-9119-dc0fc70736ec admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:15.288833 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 970/1983] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:15 2022] PUT /identity/v3/projects/5e37db690681428f8e47bc6f46b7dbd7/users/c49c8104d02f4431a94f165206717531/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:15.299500 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0146a16d-6566-48e9-98b6-555fe15cd385 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:15.300127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0146a16d-6566-48e9-98b6-555fe15cd385 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:15.300369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0146a16d-6566-48e9-98b6-555fe15cd385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:15.335847 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-0146a16d-6566-48e9-98b6-555fe15cd385 None None] MFA Rules not processed for user `c49c8104d02f4431a94f165206717531`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:15.337356 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0146a16d-6566-48e9-98b6-555fe15cd385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:15.413260 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1014/1984] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:15.424629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a38c44-5bac-481a-9119-dc0fc70736ec admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:15.426343 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-64a38c44-5bac-481a-9119-dc0fc70736ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:15.467553 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:15.521533 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a38c44-5bac-481a-9119-dc0fc70736ec admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:15.542197 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f88b9-b5ea-4491-b7ee-c6b4bd6ba5a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:15.542839 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f88b9-b5ea-4491-b7ee-c6b4bd6ba5a5 service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:15.543152 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f88b9-b5ea-4491-b7ee-c6b4bd6ba5a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:15.543430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f88b9-b5ea-4491-b7ee-c6b4bd6ba5a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:15.552807 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f88b9-b5ea-4491-b7ee-c6b4bd6ba5a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:15.554404 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7f88b9-b5ea-4491-b7ee-c6b4bd6ba5a5 service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:15.566490 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 971/1985] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:10:22.674866 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0146a16d-6566-48e9-98b6-555fe15cd385 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:22.676339 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0146a16d-6566-48e9-98b6-555fe15cd385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:22.747848 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0146a16d-6566-48e9-98b6-555fe15cd385 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:22.752495 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fa69a-33e5-4ad4-86d6-5e9b599eceab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:22.753125 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fa69a-33e5-4ad4-86d6-5e9b599eceab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:22.753405 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fa69a-33e5-4ad4-86d6-5e9b599eceab admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:22.753675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fa69a-33e5-4ad4-86d6-5e9b599eceab admin admin] PATH_INFO: `/v3/users/6080ff93f4144848b63e9a177f85fdf9` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:22.754771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663fa69a-33e5-4ad4-86d6-5e9b599eceab admin admin] RBAC: Authorizing `identity:delete_user(user_id=6080ff93f4144848b63e9a177f85fdf9)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:22.756725 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663fa69a-33e5-4ad4-86d6-5e9b599eceab admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:22.773882 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:22.812677 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-663fa69a-33e5-4ad4-86d6-5e9b599eceab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6080ff93f4144848b63e9a177f85fdf9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:22.821143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-663fa69a-33e5-4ad4-86d6-5e9b599eceab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6080ff93f4144848b63e9a177f85fdf9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:22.830141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-663fa69a-33e5-4ad4-86d6-5e9b599eceab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6080ff93f4144848b63e9a177f85fdf9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:22.830787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-663fa69a-33e5-4ad4-86d6-5e9b599eceab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6080ff93f4144848b63e9a177f85fdf9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:22.831387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-663fa69a-33e5-4ad4-86d6-5e9b599eceab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6080ff93f4144848b63e9a177f85fdf9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:22.848788 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1015/1986] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:10:22 2022] DELETE /identity/v3/users/6080ff93f4144848b63e9a177f85fdf9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:22.853165 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67c2fe-121b-4257-90ad-32be528d5056 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:22.853614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67c2fe-121b-4257-90ad-32be528d5056 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:22.853719 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67c2fe-121b-4257-90ad-32be528d5056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:22.872298 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-9c67c2fe-121b-4257-90ad-32be528d5056 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:22.873523 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-9c67c2fe-121b-4257-90ad-32be528d5056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:22.934435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 972/1987] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:10:22.944156 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fa69a-33e5-4ad4-86d6-5e9b599eceab admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:22.945556 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-663fa69a-33e5-4ad4-86d6-5e9b599eceab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:23.003415 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fa69a-33e5-4ad4-86d6-5e9b599eceab admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:23.007011 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b21a91-6e7f-432f-8c75-ca055640bbb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:23.007639 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b21a91-6e7f-432f-8c75-ca055640bbb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:23.007969 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b21a91-6e7f-432f-8c75-ca055640bbb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:23.008239 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b21a91-6e7f-432f-8c75-ca055640bbb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:23.017692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b21a91-6e7f-432f-8c75-ca055640bbb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:23.019129 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b21a91-6e7f-432f-8c75-ca055640bbb5 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:23.030096 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1016/1988] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:10:23.299034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67c2fe-121b-4257-90ad-32be528d5056 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:23.300508 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-9c67c2fe-121b-4257-90ad-32be528d5056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:23.312802 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67c2fe-121b-4257-90ad-32be528d5056 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:23.318342 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd06da6-0c03-422c-94cf-02fac59578ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:23.319059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd06da6-0c03-422c-94cf-02fac59578ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:23.319368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd06da6-0c03-422c-94cf-02fac59578ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:23.319669 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd06da6-0c03-422c-94cf-02fac59578ab admin admin] PATH_INFO: `/v3/projects/0ae24aa029a1481087bd9e0387035272` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:23.320797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd06da6-0c03-422c-94cf-02fac59578ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ae24aa029a1481087bd9e0387035272)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:23.325125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd06da6-0c03-422c-94cf-02fac59578ab admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:23.388058 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-2fd06da6-0c03-422c-94cf-02fac59578ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ae24aa029a1481087bd9e0387035272', 'request_id': 'req-2fd06da6-0c03-422c-94cf-02fac59578ab'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:23.395108 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-2fd06da6-0c03-422c-94cf-02fac59578ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ae24aa029a1481087bd9e0387035272', 'request_id': 'req-2fd06da6-0c03-422c-94cf-02fac59578ab'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:23.401379 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-2fd06da6-0c03-422c-94cf-02fac59578ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ae24aa029a1481087bd9e0387035272', 'request_id': 'req-2fd06da6-0c03-422c-94cf-02fac59578ab'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:23.414640 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-2fd06da6-0c03-422c-94cf-02fac59578ab admin admin] The token cache is being invalidate because project 0ae24aa029a1481087bd9e0387035272 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:10:23.414929 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-2fd06da6-0c03-422c-94cf-02fac59578ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:23.415594 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-2fd06da6-0c03-422c-94cf-02fac59578ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:23.417132 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 973/1989] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:10:23 2022] DELETE /identity/v3/projects/0ae24aa029a1481087bd9e0387035272 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:23.421223 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b21a91-6e7f-432f-8c75-ca055640bbb5 service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:23.431647 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b21a91-6e7f-432f-8c75-ca055640bbb5 service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:23.494531 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d26d4-1847-4a61-911e-fbc3448d1e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:23.495253 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d26d4-1847-4a61-911e-fbc3448d1e90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:23.495535 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d26d4-1847-4a61-911e-fbc3448d1e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:23.495808 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d26d4-1847-4a61-911e-fbc3448d1e90 admin admin] PATH_INFO: `/v3/users/55e167f8d9224fe698fa9c8a05e947ec` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:23.496931 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d26d4-1847-4a61-911e-fbc3448d1e90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55e167f8d9224fe698fa9c8a05e947ec)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:23.498899 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d26d4-1847-4a61-911e-fbc3448d1e90 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:23.544055 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:23.556065 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-464d26d4-1847-4a61-911e-fbc3448d1e90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55e167f8d9224fe698fa9c8a05e947ec'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:23.565239 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-464d26d4-1847-4a61-911e-fbc3448d1e90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55e167f8d9224fe698fa9c8a05e947ec'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:23.578105 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-464d26d4-1847-4a61-911e-fbc3448d1e90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55e167f8d9224fe698fa9c8a05e947ec'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:23.578940 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-464d26d4-1847-4a61-911e-fbc3448d1e90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55e167f8d9224fe698fa9c8a05e947ec'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:23.579678 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-464d26d4-1847-4a61-911e-fbc3448d1e90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55e167f8d9224fe698fa9c8a05e947ec'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:23.599958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1017/1990] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:10:23 2022] DELETE /identity/v3/users/55e167f8d9224fe698fa9c8a05e947ec => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:23.956558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd06da6-0c03-422c-94cf-02fac59578ab admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:23.958122 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-2fd06da6-0c03-422c-94cf-02fac59578ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:23.979040 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd06da6-0c03-422c-94cf-02fac59578ab admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:23.997972 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d26d4-1847-4a61-911e-fbc3448d1e90 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:24.000064 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-464d26d4-1847-4a61-911e-fbc3448d1e90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:24.033432 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:24.074267 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d5957-456b-46ee-9735-267e0f87564a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:24.074949 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d5957-456b-46ee-9735-267e0f87564a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:24.075264 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d5957-456b-46ee-9735-267e0f87564a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:24.075544 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d5957-456b-46ee-9735-267e0f87564a admin admin] PATH_INFO: `/v3/users/c49c8104d02f4431a94f165206717531` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:24.076702 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d5957-456b-46ee-9735-267e0f87564a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c49c8104d02f4431a94f165206717531)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:24.079180 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d5957-456b-46ee-9735-267e0f87564a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:24.094234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d26d4-1847-4a61-911e-fbc3448d1e90 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:24.102673 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38617ce0-ff58-4193-8cc5-97420ee4ef8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:24.103552 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38617ce0-ff58-4193-8cc5-97420ee4ef8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:24.103837 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38617ce0-ff58-4193-8cc5-97420ee4ef8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:24.104373 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38617ce0-ff58-4193-8cc5-97420ee4ef8b admin admin] PATH_INFO: `/v3/projects/e41533102c2d49d38c3337a960e8bbaf` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:24.106195 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38617ce0-ff58-4193-8cc5-97420ee4ef8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e41533102c2d49d38c3337a960e8bbaf)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:24.108263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38617ce0-ff58-4193-8cc5-97420ee4ef8b admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:24.144213 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-715d5957-456b-46ee-9735-267e0f87564a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c49c8104d02f4431a94f165206717531'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:24.145174 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-715d5957-456b-46ee-9735-267e0f87564a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c49c8104d02f4431a94f165206717531'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:24.162036 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-715d5957-456b-46ee-9735-267e0f87564a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c49c8104d02f4431a94f165206717531'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:24.170661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-715d5957-456b-46ee-9735-267e0f87564a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c49c8104d02f4431a94f165206717531'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:24.171958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-715d5957-456b-46ee-9735-267e0f87564a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c49c8104d02f4431a94f165206717531'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:24.180839 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-38617ce0-ff58-4193-8cc5-97420ee4ef8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e41533102c2d49d38c3337a960e8bbaf', 'request_id': 'req-38617ce0-ff58-4193-8cc5-97420ee4ef8b'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:24.187862 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 974/1991] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:10:23 2022] DELETE /identity/v3/users/c49c8104d02f4431a94f165206717531 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:24.192612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5e101-3ec5-4b23-93d4-db5248d755cc None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:24.192961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5e101-3ec5-4b23-93d4-db5248d755cc None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:24.193309 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5e101-3ec5-4b23-93d4-db5248d755cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:24.193597 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-38617ce0-ff58-4193-8cc5-97420ee4ef8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e41533102c2d49d38c3337a960e8bbaf', 'request_id': 'req-38617ce0-ff58-4193-8cc5-97420ee4ef8b'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:24.202758 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-38617ce0-ff58-4193-8cc5-97420ee4ef8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e41533102c2d49d38c3337a960e8bbaf', 'request_id': 'req-38617ce0-ff58-4193-8cc5-97420ee4ef8b'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:24.215105 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-38617ce0-ff58-4193-8cc5-97420ee4ef8b admin admin] The token cache is being invalidate because project e41533102c2d49d38c3337a960e8bbaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:10:24.215499 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-38617ce0-ff58-4193-8cc5-97420ee4ef8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:24.216212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-38617ce0-ff58-4193-8cc5-97420ee4ef8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:24.216592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-c0b5e101-3ec5-4b23-93d4-db5248d755cc None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:24.217798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c0b5e101-3ec5-4b23-93d4-db5248d755cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:24.218264 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1018/1992] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:10:23 2022] DELETE /identity/v3/projects/e41533102c2d49d38c3337a960e8bbaf => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:10:24.222916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a865a08-8879-4af6-bc20-5e20fdb4d4ba None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:24.223240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a865a08-8879-4af6-bc20-5e20fdb4d4ba None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:24.223528 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a865a08-8879-4af6-bc20-5e20fdb4d4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:24.244842 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-9a865a08-8879-4af6-bc20-5e20fdb4d4ba None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:24.246048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-9a865a08-8879-4af6-bc20-5e20fdb4d4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:24.297096 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:24.305892 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 975/1993] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:10:24.316031 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5e101-3ec5-4b23-93d4-db5248d755cc None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:24.317830 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c0b5e101-3ec5-4b23-93d4-db5248d755cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:24.355122 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1019/1994] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:10:24.366710 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2337095-6105-4c50-bb5a-0dd7a2b4a196 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:24.367290 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2337095-6105-4c50-bb5a-0dd7a2b4a196 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:24.367565 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2337095-6105-4c50-bb5a-0dd7a2b4a196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:24.407563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-f2337095-6105-4c50-bb5a-0dd7a2b4a196 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:24.408838 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f2337095-6105-4c50-bb5a-0dd7a2b4a196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:24.415967 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5e101-3ec5-4b23-93d4-db5248d755cc None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:24.418874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1020/1995] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:24.423468 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2337095-6105-4c50-bb5a-0dd7a2b4a196 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:24.424988 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e66ee8-8c33-42e7-939f-c8cce0bb827c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:24.425573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e66ee8-8c33-42e7-939f-c8cce0bb827c service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:24.425862 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e66ee8-8c33-42e7-939f-c8cce0bb827c service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:24.426139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e66ee8-8c33-42e7-939f-c8cce0bb827c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:24.435040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2337095-6105-4c50-bb5a-0dd7a2b4a196 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:24.435941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e66ee8-8c33-42e7-939f-c8cce0bb827c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:24.437380 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e66ee8-8c33-42e7-939f-c8cce0bb827c service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:24.440451 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9ebb5-40a4-41c1-8a77-da593032865f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:24.441050 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9ebb5-40a4-41c1-8a77-da593032865f admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:24.441330 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9ebb5-40a4-41c1-8a77-da593032865f admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:24.441603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9ebb5-40a4-41c1-8a77-da593032865f admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:24.442128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:24.442238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9ebb5-40a4-41c1-8a77-da593032865f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:10:24.442530 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9ebb5-40a4-41c1-8a77-da593032865f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:24.444178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9ebb5-40a4-41c1-8a77-da593032865f admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:24.449962 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 976/1996] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:10:24.456457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1021/1997] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:10:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:10:24.460782 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e66ee8-8c33-42e7-939f-c8cce0bb827c service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:24.470712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e66ee8-8c33-42e7-939f-c8cce0bb827c service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:24.475542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dde71d-4e1c-446b-815c-89d3fad43247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:24.476141 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dde71d-4e1c-446b-815c-89d3fad43247 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:24.476451 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dde71d-4e1c-446b-815c-89d3fad43247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:24.476726 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dde71d-4e1c-446b-815c-89d3fad43247 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:24.477384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dde71d-4e1c-446b-815c-89d3fad43247 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:24.479558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dde71d-4e1c-446b-815c-89d3fad43247 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:24.507787 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 977/1998] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:10:24 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:24.511983 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9ebb5-40a4-41c1-8a77-da593032865f admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:24.522888 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9ebb5-40a4-41c1-8a77-da593032865f admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:24.584797 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c4a61-a67e-414a-bfa9-4f157dad6881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:24.585489 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c4a61-a67e-414a-bfa9-4f157dad6881 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:24.585802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c4a61-a67e-414a-bfa9-4f157dad6881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:24.586074 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c4a61-a67e-414a-bfa9-4f157dad6881 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:24.586706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c4a61-a67e-414a-bfa9-4f157dad6881 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:24.588663 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c4a61-a67e-414a-bfa9-4f157dad6881 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:24.614291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1022/1999] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:10:24 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:24.618826 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dde71d-4e1c-446b-815c-89d3fad43247 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:24.635530 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dde71d-4e1c-446b-815c-89d3fad43247 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:24.642101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a0637-d14e-42c6-a05e-2eb94fd67a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:24.642821 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a0637-d14e-42c6-a05e-2eb94fd67a1d admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:24.643235 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a0637-d14e-42c6-a05e-2eb94fd67a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:24.643600 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a0637-d14e-42c6-a05e-2eb94fd67a1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:24.651135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a0637-d14e-42c6-a05e-2eb94fd67a1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:24.651135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a0637-d14e-42c6-a05e-2eb94fd67a1d admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:24.653508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240d905-1687-49d9-898f-cb88e8344ed5 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:24.653797 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240d905-1687-49d9-898f-cb88e8344ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:24.654076 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240d905-1687-49d9-898f-cb88e8344ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:24.661555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 978/2000] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:24.662939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a0637-d14e-42c6-a05e-2eb94fd67a1d admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:24.672850 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a0637-d14e-42c6-a05e-2eb94fd67a1d admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:24.674577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-e240d905-1687-49d9-898f-cb88e8344ed5 None None] MFA Rules not processed for user `66133313929d4c0f98ab21b1ea336456`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:24.676296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-e240d905-1687-49d9-898f-cb88e8344ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:24.678328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2259ae-d9d7-40c4-bc9a-39c5fd2c20c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:24.679028 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2259ae-d9d7-40c4-bc9a-39c5fd2c20c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:24.679841 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2259ae-d9d7-40c4-bc9a-39c5fd2c20c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:24.679841 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2259ae-d9d7-40c4-bc9a-39c5fd2c20c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:24.680524 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2259ae-d9d7-40c4-bc9a-39c5fd2c20c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:24.682178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2259ae-d9d7-40c4-bc9a-39c5fd2c20c0 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:24.698027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 979/2001] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:24.698886 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2259ae-d9d7-40c4-bc9a-39c5fd2c20c0 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:24.711319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2259ae-d9d7-40c4-bc9a-39c5fd2c20c0 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:24.716518 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d02bf-1f68-49ab-b3ea-c04b93f6473d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:24.717190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d02bf-1f68-49ab-b3ea-c04b93f6473d admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:24.717472 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d02bf-1f68-49ab-b3ea-c04b93f6473d admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:24.717738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d02bf-1f68-49ab-b3ea-c04b93f6473d admin admin] PATH_INFO: `/v3/projects/5c0195da1ae344f3abc909d9e27f5386/users/95cf77b17c404175ae50584d32852aa5/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:24.741106 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d02bf-1f68-49ab-b3ea-c04b93f6473d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c0195da1ae344f3abc909d9e27f5386, user_id=95cf77b17c404175ae50584d32852aa5, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:24.744118 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d02bf-1f68-49ab-b3ea-c04b93f6473d admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:24.766050 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1023/2002] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:24.767008 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240d905-1687-49d9-898f-cb88e8344ed5 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:24.768262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-e240d905-1687-49d9-898f-cb88e8344ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:24.772117 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 980/2003] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:24 2022] PUT /identity/v3/projects/5c0195da1ae344f3abc909d9e27f5386/users/95cf77b17c404175ae50584d32852aa5/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:24.784592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fb3b6-01e2-4985-a199-0c4477bd4443 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:24.785186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fb3b6-01e2-4985-a199-0c4477bd4443 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:24.785618 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fb3b6-01e2-4985-a199-0c4477bd4443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:24.804676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240d905-1687-49d9-898f-cb88e8344ed5 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:24.819971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-d92fb3b6-01e2-4985-a199-0c4477bd4443 None None] MFA Rules not processed for user `95cf77b17c404175ae50584d32852aa5`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:24.821206 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-d92fb3b6-01e2-4985-a199-0c4477bd4443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:24.875002 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ca98b-0f5a-4808-a466-597dc85bd0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:24.876007 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ca98b-0f5a-4808-a466-597dc85bd0b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:24.876355 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ca98b-0f5a-4808-a466-597dc85bd0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:24.876637 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ca98b-0f5a-4808-a466-597dc85bd0b7 admin admin] PATH_INFO: `/v3/projects/5e37db690681428f8e47bc6f46b7dbd7` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:24.877750 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ca98b-0f5a-4808-a466-597dc85bd0b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e37db690681428f8e47bc6f46b7dbd7)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:24.879857 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ca98b-0f5a-4808-a466-597dc85bd0b7 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:24.883123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:24.893344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 981/2004] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:10:24.897720 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fb3b6-01e2-4985-a199-0c4477bd4443 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:24.910869 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fb3b6-01e2-4985-a199-0c4477bd4443 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:24.918932 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbaec17-abc1-4e6f-8266-af78181483d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:24.919615 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbaec17-abc1-4e6f-8266-af78181483d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:24.919894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbaec17-abc1-4e6f-8266-af78181483d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:24.920251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbaec17-abc1-4e6f-8266-af78181483d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:24.920895 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbaec17-abc1-4e6f-8266-af78181483d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:24.923386 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbaec17-abc1-4e6f-8266-af78181483d6 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:24.937675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-2a6ca98b-0f5a-4808-a466-597dc85bd0b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e37db690681428f8e47bc6f46b7dbd7', 'request_id': 'req-2a6ca98b-0f5a-4808-a466-597dc85bd0b7'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:24.947008 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-2a6ca98b-0f5a-4808-a466-597dc85bd0b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e37db690681428f8e47bc6f46b7dbd7', 'request_id': 'req-2a6ca98b-0f5a-4808-a466-597dc85bd0b7'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:24.954474 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-2a6ca98b-0f5a-4808-a466-597dc85bd0b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e37db690681428f8e47bc6f46b7dbd7', 'request_id': 'req-2a6ca98b-0f5a-4808-a466-597dc85bd0b7'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:24.957150 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 982/2005] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:10:24 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:10:24.961003 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbaec17-abc1-4e6f-8266-af78181483d6 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:24.966877 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-2a6ca98b-0f5a-4808-a466-597dc85bd0b7 admin admin] The token cache is being invalidate because project 5e37db690681428f8e47bc6f46b7dbd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:10:24.967245 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-2a6ca98b-0f5a-4808-a466-597dc85bd0b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:24.968084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-2a6ca98b-0f5a-4808-a466-597dc85bd0b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:24.970070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1024/2006] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:10:24 2022] DELETE /identity/v3/projects/5e37db690681428f8e47bc6f46b7dbd7 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:24.974698 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb81dc3-ea68-42c9-a2b3-a9601ea9e5e4 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:24.975010 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb81dc3-ea68-42c9-a2b3-a9601ea9e5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:24.975461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb81dc3-ea68-42c9-a2b3-a9601ea9e5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:25.001484 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-5cb81dc3-ea68-42c9-a2b3-a9601ea9e5e4 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:25.003045 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5cb81dc3-ea68-42c9-a2b3-a9601ea9e5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:25.028934 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:25.033505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbaec17-abc1-4e6f-8266-af78181483d6 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:25.039846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc09e4e-fec4-4159-9ae6-a23c0a611fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:25.040601 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc09e4e-fec4-4159-9ae6-a23c0a611fc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:25.040967 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc09e4e-fec4-4159-9ae6-a23c0a611fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:25.041344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc09e4e-fec4-4159-9ae6-a23c0a611fc2 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:25.042100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc09e4e-fec4-4159-9ae6-a23c0a611fc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:25.044164 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc09e4e-fec4-4159-9ae6-a23c0a611fc2 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:25.067759 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 983/2007] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:10:24 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:10:25.071582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc09e4e-fec4-4159-9ae6-a23c0a611fc2 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:25.072242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1025/2008] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:10:25.076238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-2fc09e4e-fec4-4159-9ae6-a23c0a611fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:25.081891 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5bfbc-e003-4717-8af5-6c158b7d7f49 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:25.082289 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5bfbc-e003-4717-8af5-6c158b7d7f49 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:25.082662 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca5bfbc-e003-4717-8af5-6c158b7d7f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:25.089885 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc09e4e-fec4-4159-9ae6-a23c0a611fc2 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:25.094794 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3625a-c878-4397-acc1-6f30a7754ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:25.095432 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3625a-c878-4397-acc1-6f30a7754ffa admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:25.095744 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3625a-c878-4397-acc1-6f30a7754ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:25.096014 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3625a-c878-4397-acc1-6f30a7754ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:25.096718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b3625a-c878-4397-acc1-6f30a7754ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:25.098339 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b3625a-c878-4397-acc1-6f30a7754ffa admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:25.105144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-cca5bfbc-e003-4717-8af5-6c158b7d7f49 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:25.106565 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-cca5bfbc-e003-4717-8af5-6c158b7d7f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:25.109118 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 984/2009] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:10:25.113943 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3625a-c878-4397-acc1-6f30a7754ffa admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:25.117303 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1026/2010] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:10:25.121309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5bfbc-e003-4717-8af5-6c158b7d7f49 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:25.124513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3625a-c878-4397-acc1-6f30a7754ffa admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:25.130501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcac4b8-3c20-48c3-bd46-e04d7273b87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:25.131393 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcac4b8-3c20-48c3-bd46-e04d7273b87d admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:25.131752 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcac4b8-3c20-48c3-bd46-e04d7273b87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:25.131871 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca5bfbc-e003-4717-8af5-6c158b7d7f49 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:25.132302 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcac4b8-3c20-48c3-bd46-e04d7273b87d admin admin] PATH_INFO: `/v3/projects/e1a4a63a68ea4ca097748f376b77dc03/users/7ee85b841254413b869acf7de66563ab/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:25.138023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e09ebf-f401-4d3b-807b-30ef483ad780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:25.138890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e09ebf-f401-4d3b-807b-30ef483ad780 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:25.139381 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e09ebf-f401-4d3b-807b-30ef483ad780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:25.139614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e09ebf-f401-4d3b-807b-30ef483ad780 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:25.140334 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e09ebf-f401-4d3b-807b-30ef483ad780 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:10:25.140701 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e09ebf-f401-4d3b-807b-30ef483ad780 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:25.142564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e09ebf-f401-4d3b-807b-30ef483ad780 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:25.148461 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcac4b8-3c20-48c3-bd46-e04d7273b87d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1a4a63a68ea4ca097748f376b77dc03, user_id=7ee85b841254413b869acf7de66563ab, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:25.151290 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcac4b8-3c20-48c3-bd46-e04d7273b87d admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:25.156177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1027/2011] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:10:25.160697 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e09ebf-f401-4d3b-807b-30ef483ad780 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:25.165985 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 985/2012] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:25 2022] PUT /identity/v3/projects/e1a4a63a68ea4ca097748f376b77dc03/users/7ee85b841254413b869acf7de66563ab/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:10:25.170286 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcac4b8-3c20-48c3-bd46-e04d7273b87d admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:25.171307 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e09ebf-f401-4d3b-807b-30ef483ad780 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:25.236049 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e17681-c01e-4c0a-8729-981e44660a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:25.236657 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e17681-c01e-4c0a-8729-981e44660a7d admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:25.236938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e17681-c01e-4c0a-8729-981e44660a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:25.237210 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e17681-c01e-4c0a-8729-981e44660a7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:25.237834 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e17681-c01e-4c0a-8729-981e44660a7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:25.240329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e17681-c01e-4c0a-8729-981e44660a7d admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:25.245022 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcac4b8-3c20-48c3-bd46-e04d7273b87d admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:25.251384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5306b9-31cd-432c-9c47-7acb2405a8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:25.252090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5306b9-31cd-432c-9c47-7acb2405a8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:25.252397 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5306b9-31cd-432c-9c47-7acb2405a8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:25.252751 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5306b9-31cd-432c-9c47-7acb2405a8e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:25.253346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5306b9-31cd-432c-9c47-7acb2405a8e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:25.254962 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5306b9-31cd-432c-9c47-7acb2405a8e7 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:25.263923 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:25.269099 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 986/2013] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:25.273648 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5306b9-31cd-432c-9c47-7acb2405a8e7 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:25.280914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1028/2014] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:10:25 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:10:25.283516 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e17681-c01e-4c0a-8729-981e44660a7d admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:25.286245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5306b9-31cd-432c-9c47-7acb2405a8e7 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:25.358536 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e17681-c01e-4c0a-8729-981e44660a7d admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:25.364023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834232bf-f37c-42f9-bc30-b00741f9e3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:25.364633 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834232bf-f37c-42f9-bc30-b00741f9e3bb admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:25.364914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834232bf-f37c-42f9-bc30-b00741f9e3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:25.365186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834232bf-f37c-42f9-bc30-b00741f9e3bb admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:25.365823 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834232bf-f37c-42f9-bc30-b00741f9e3bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:25.367838 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834232bf-f37c-42f9-bc30-b00741f9e3bb admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:25.371350 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ec7c1-fbd0-4179-8d26-15cf5b2fd0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:25.372231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ec7c1-fbd0-4179-8d26-15cf5b2fd0e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:25.372676 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ec7c1-fbd0-4179-8d26-15cf5b2fd0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:25.373096 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ec7c1-fbd0-4179-8d26-15cf5b2fd0e2 admin admin] PATH_INFO: `/v3/projects/e1a4a63a68ea4ca097748f376b77dc03/users/7ee85b841254413b869acf7de66563ab/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:25.375386 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ec7c1-fbd0-4179-8d26-15cf5b2fd0e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1a4a63a68ea4ca097748f376b77dc03, user_id=7ee85b841254413b869acf7de66563ab, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:25.378531 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ec7c1-fbd0-4179-8d26-15cf5b2fd0e2 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:25.389998 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1029/2015] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:10:25 2022] POST /identity/v3/users => generated 283 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:25.402126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 987/2016] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:25 2022] PUT /identity/v3/projects/e1a4a63a68ea4ca097748f376b77dc03/users/7ee85b841254413b869acf7de66563ab/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:25.403594 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ec7c1-fbd0-4179-8d26-15cf5b2fd0e2 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:25.404104 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54dd9a4-479e-4605-a149-99f96f07979e None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:25.404485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54dd9a4-479e-4605-a149-99f96f07979e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:25.406018 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54dd9a4-479e-4605-a149-99f96f07979e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:25.454553 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-a54dd9a4-479e-4605-a149-99f96f07979e None None] MFA Rules not processed for user `7ee85b841254413b869acf7de66563ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:25.458464 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a54dd9a4-479e-4605-a149-99f96f07979e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:25.488598 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:25.511041 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ec7c1-fbd0-4179-8d26-15cf5b2fd0e2 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:25.516363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da21315-7af1-45f6-94a4-7d4db295a377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:25.516977 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da21315-7af1-45f6-94a4-7d4db295a377 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:25.517309 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da21315-7af1-45f6-94a4-7d4db295a377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:25.517589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da21315-7af1-45f6-94a4-7d4db295a377 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:25.519018 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da21315-7af1-45f6-94a4-7d4db295a377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:25.524129 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da21315-7af1-45f6-94a4-7d4db295a377 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:25.538672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 988/2017] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:25.544637 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da21315-7af1-45f6-94a4-7d4db295a377 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:25.565820 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1030/2018] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:25.569217 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da21315-7af1-45f6-94a4-7d4db295a377 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:25.576882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a50a17-e341-490d-83b8-1c7874ff5fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:25.577490 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a50a17-e341-490d-83b8-1c7874ff5fab admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:25.578428 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54dd9a4-479e-4605-a149-99f96f07979e None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:25.580611 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a54dd9a4-479e-4605-a149-99f96f07979e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:25.581731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a50a17-e341-490d-83b8-1c7874ff5fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:25.582035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a50a17-e341-490d-83b8-1c7874ff5fab admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:25.584083 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a50a17-e341-490d-83b8-1c7874ff5fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:25.589101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a50a17-e341-490d-83b8-1c7874ff5fab admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:25.609036 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 989/2019] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:25 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:25.612129 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a50a17-e341-490d-83b8-1c7874ff5fab admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:25.623442 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a50a17-e341-490d-83b8-1c7874ff5fab admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:25.629042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2fe986-e2ff-4496-9be4-9ad43aa36f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:25.630081 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fe986-e2ff-4496-9be4-9ad43aa36f11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:25.630571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fe986-e2ff-4496-9be4-9ad43aa36f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:25.631030 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fe986-e2ff-4496-9be4-9ad43aa36f11 admin admin] PATH_INFO: `/v3/projects/6b680ed7148a494e94972833dc2348e2/users/1a4e5504823747fa96216667129b2107/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:25.657361 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2fe986-e2ff-4496-9be4-9ad43aa36f11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b680ed7148a494e94972833dc2348e2, user_id=1a4e5504823747fa96216667129b2107, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:25.661837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2fe986-e2ff-4496-9be4-9ad43aa36f11 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:25.669073 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54dd9a4-479e-4605-a149-99f96f07979e None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:25.677019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a9781-103d-4581-b5bc-96431b471201 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:25.678119 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a9781-103d-4581-b5bc-96431b471201 service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:25.678713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a9781-103d-4581-b5bc-96431b471201 service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:25.679413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a9781-103d-4581-b5bc-96431b471201 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:25.685228 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 990/2020] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:25 2022] PUT /identity/v3/projects/6b680ed7148a494e94972833dc2348e2/users/1a4e5504823747fa96216667129b2107/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:25.693619 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a9781-103d-4581-b5bc-96431b471201 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:25.696349 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a1310-11ce-427b-a7d5-b56667ba57d0 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:25.696484 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a9781-103d-4581-b5bc-96431b471201 service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:25.697058 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a1310-11ce-427b-a7d5-b56667ba57d0 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:25.697616 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a1310-11ce-427b-a7d5-b56667ba57d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:25.720394 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1031/2021] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:10:25.736171 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-526a1310-11ce-427b-a7d5-b56667ba57d0 None None] MFA Rules not processed for user `1a4e5504823747fa96216667129b2107`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:25.738143 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-526a1310-11ce-427b-a7d5-b56667ba57d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:25.740159 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a9781-103d-4581-b5bc-96431b471201 service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:25.751948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a9781-103d-4581-b5bc-96431b471201 service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:25.776773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:25.869414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160a130-cdb3-4728-8d28-dd24a4fcbeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:25.870258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160a130-cdb3-4728-8d28-dd24a4fcbeca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:25.870616 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160a130-cdb3-4728-8d28-dd24a4fcbeca admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:25.870964 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160a130-cdb3-4728-8d28-dd24a4fcbeca admin admin] PATH_INFO: `/v3/users/95cf77b17c404175ae50584d32852aa5` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:25.872228 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160a130-cdb3-4728-8d28-dd24a4fcbeca admin admin] RBAC: Authorizing `identity:delete_user(user_id=95cf77b17c404175ae50584d32852aa5)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:25.874261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160a130-cdb3-4728-8d28-dd24a4fcbeca admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:25.878332 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 991/2022] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:10:25.882743 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a1310-11ce-427b-a7d5-b56667ba57d0 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:25.899902 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a1310-11ce-427b-a7d5-b56667ba57d0 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:25.915793 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2a5a8-e957-4831-aa28-9d9872877fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:25.917294 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2a5a8-e957-4831-aa28-9d9872877fec admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:25.917294 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2a5a8-e957-4831-aa28-9d9872877fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:25.917294 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2a5a8-e957-4831-aa28-9d9872877fec admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:25.917866 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb2a5a8-e957-4831-aa28-9d9872877fec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:25.920012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb2a5a8-e957-4831-aa28-9d9872877fec admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:25.950442 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-c160a130-cdb3-4728-8d28-dd24a4fcbeca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95cf77b17c404175ae50584d32852aa5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:25.959829 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 992/2023] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:10:25 2022] POST /identity/v3/projects => generated 363 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:10:25.961125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2a5a8-e957-4831-aa28-9d9872877fec admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:25.962097 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-c160a130-cdb3-4728-8d28-dd24a4fcbeca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95cf77b17c404175ae50584d32852aa5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:25.972915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-c160a130-cdb3-4728-8d28-dd24a4fcbeca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95cf77b17c404175ae50584d32852aa5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:25.974244 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-c160a130-cdb3-4728-8d28-dd24a4fcbeca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95cf77b17c404175ae50584d32852aa5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:25.974678 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-c160a130-cdb3-4728-8d28-dd24a4fcbeca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95cf77b17c404175ae50584d32852aa5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:25.996366 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1032/2024] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:10:25 2022] DELETE /identity/v3/users/95cf77b17c404175ae50584d32852aa5 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:26.001300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a940f-e93c-4d1f-a4b0-e93e1b68c1cb None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:26.001595 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a940f-e93c-4d1f-a4b0-e93e1b68c1cb None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:26.002161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a940f-e93c-4d1f-a4b0-e93e1b68c1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:26.024496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-006a940f-e93c-4d1f-a4b0-e93e1b68c1cb None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:26.025774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-006a940f-e93c-4d1f-a4b0-e93e1b68c1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:26.035539 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2a5a8-e957-4831-aa28-9d9872877fec admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:26.051696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:26.093665 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1033/2025] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:26.104262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a940f-e93c-4d1f-a4b0-e93e1b68c1cb None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:26.105656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-006a940f-e93c-4d1f-a4b0-e93e1b68c1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:26.130432 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9dafc-b034-41fc-a49d-9094d327f397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:26.131279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9dafc-b034-41fc-a49d-9094d327f397 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:26.131451 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9dafc-b034-41fc-a49d-9094d327f397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:26.131697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9dafc-b034-41fc-a49d-9094d327f397 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:26.132714 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9dafc-b034-41fc-a49d-9094d327f397 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:26.134665 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9dafc-b034-41fc-a49d-9094d327f397 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:26.157451 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:26.160196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 993/2026] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:10:25 2022] POST /identity/v3/users => generated 288 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:26.164358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9dafc-b034-41fc-a49d-9094d327f397 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:26.166365 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-bcb9dafc-b034-41fc-a49d-9094d327f397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:26.179289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9dafc-b034-41fc-a49d-9094d327f397 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:26.182435 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a940f-e93c-4d1f-a4b0-e93e1b68c1cb None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:26.185584 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75332504-1e47-4b60-a343-5d5e6e1adad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:26.186136 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75332504-1e47-4b60-a343-5d5e6e1adad7 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:26.186332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba5039-5cc0-4ba2-8b06-3cee7b8f8ac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:26.186442 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75332504-1e47-4b60-a343-5d5e6e1adad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:26.186679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75332504-1e47-4b60-a343-5d5e6e1adad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:26.186956 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba5039-5cc0-4ba2-8b06-3cee7b8f8ac5 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:26.187214 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba5039-5cc0-4ba2-8b06-3cee7b8f8ac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:26.187648 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75332504-1e47-4b60-a343-5d5e6e1adad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:26.187792 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ba5039-5cc0-4ba2-8b06-3cee7b8f8ac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:26.189592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75332504-1e47-4b60-a343-5d5e6e1adad7 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:26.197994 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba5039-5cc0-4ba2-8b06-3cee7b8f8ac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:26.199724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ba5039-5cc0-4ba2-8b06-3cee7b8f8ac5 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:26.201140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 994/2027] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:26.205969 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75332504-1e47-4b60-a343-5d5e6e1adad7 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:26.211031 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1034/2028] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:10:26.216122 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75332504-1e47-4b60-a343-5d5e6e1adad7 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:26.221154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b482ffc-55c4-4e69-8831-b8136e79ea69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:26.221813 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b482ffc-55c4-4e69-8831-b8136e79ea69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:26.222112 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b482ffc-55c4-4e69-8831-b8136e79ea69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:26.222375 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b482ffc-55c4-4e69-8831-b8136e79ea69 admin admin] PATH_INFO: `/v3/projects/611bc08405cd4af1ba87230bb1c704c1/users/a1f288cc47c544c6be3693a589784c6f/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:26.239779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b482ffc-55c4-4e69-8831-b8136e79ea69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=611bc08405cd4af1ba87230bb1c704c1, user_id=a1f288cc47c544c6be3693a589784c6f, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:26.243853 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b482ffc-55c4-4e69-8831-b8136e79ea69 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:26.265486 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 995/2029] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:26 2022] PUT /identity/v3/projects/611bc08405cd4af1ba87230bb1c704c1/users/a1f288cc47c544c6be3693a589784c6f/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:26.266626 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba5039-5cc0-4ba2-8b06-3cee7b8f8ac5 service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:26.281173 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ba5039-5cc0-4ba2-8b06-3cee7b8f8ac5 service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:26.380222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447dc3b7-1522-4c97-979e-ef53e11dde8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:26.381980 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447dc3b7-1522-4c97-979e-ef53e11dde8d admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:26.381980 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447dc3b7-1522-4c97-979e-ef53e11dde8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:26.382180 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447dc3b7-1522-4c97-979e-ef53e11dde8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:26.382837 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447dc3b7-1522-4c97-979e-ef53e11dde8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:26.385393 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447dc3b7-1522-4c97-979e-ef53e11dde8d admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:26.410033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1035/2030] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:26.414553 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b482ffc-55c4-4e69-8831-b8136e79ea69 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:26.428802 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b482ffc-55c4-4e69-8831-b8136e79ea69 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:26.433933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491879c-1a3e-46a8-8ce6-4bcf9987573d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:26.434601 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491879c-1a3e-46a8-8ce6-4bcf9987573d admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:26.434882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491879c-1a3e-46a8-8ce6-4bcf9987573d admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:26.435240 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491879c-1a3e-46a8-8ce6-4bcf9987573d admin admin] PATH_INFO: `/v3/projects/611bc08405cd4af1ba87230bb1c704c1/users/a1f288cc47c544c6be3693a589784c6f/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:26.437254 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491879c-1a3e-46a8-8ce6-4bcf9987573d admin admin] RBAC: Authorizing `identity:create_grant(project_id=611bc08405cd4af1ba87230bb1c704c1, user_id=a1f288cc47c544c6be3693a589784c6f, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:26.440144 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491879c-1a3e-46a8-8ce6-4bcf9987573d admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:26.451100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 996/2031] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:26 2022] PUT /identity/v3/projects/611bc08405cd4af1ba87230bb1c704c1/users/a1f288cc47c544c6be3693a589784c6f/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:10:26.469280 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610316d-fbfb-40fc-9122-f1512cca019d None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:26.470308 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610316d-fbfb-40fc-9122-f1512cca019d None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:26.470786 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610316d-fbfb-40fc-9122-f1512cca019d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:26.508887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-0610316d-fbfb-40fc-9122-f1512cca019d None None] MFA Rules not processed for user `a1f288cc47c544c6be3693a589784c6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:26.510325 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0610316d-fbfb-40fc-9122-f1512cca019d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:26.574554 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491879c-1a3e-46a8-8ce6-4bcf9987573d admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:26.576777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-9491879c-1a3e-46a8-8ce6-4bcf9987573d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:26.612943 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1036/2032] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:26.635553 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610316d-fbfb-40fc-9122-f1512cca019d None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:26.636960 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0610316d-fbfb-40fc-9122-f1512cca019d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:26.662964 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491879c-1a3e-46a8-8ce6-4bcf9987573d admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:26.667806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddccd39-1a12-4d1e-8b03-5f2beaa28e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:26.668474 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddccd39-1a12-4d1e-8b03-5f2beaa28e24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:26.668758 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddccd39-1a12-4d1e-8b03-5f2beaa28e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:26.669065 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddccd39-1a12-4d1e-8b03-5f2beaa28e24 admin admin] PATH_INFO: `/v3/projects/5c0195da1ae344f3abc909d9e27f5386` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:26.670360 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddccd39-1a12-4d1e-8b03-5f2beaa28e24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c0195da1ae344f3abc909d9e27f5386)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:26.672323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddccd39-1a12-4d1e-8b03-5f2beaa28e24 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:26.695251 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610316d-fbfb-40fc-9122-f1512cca019d None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:26.699404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93739fb0-9ecc-4bf0-9929-efb9d7fc9035 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:26.700003 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93739fb0-9ecc-4bf0-9929-efb9d7fc9035 service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:26.700324 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93739fb0-9ecc-4bf0-9929-efb9d7fc9035 service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:26.700549 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93739fb0-9ecc-4bf0-9929-efb9d7fc9035 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:26.711558 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93739fb0-9ecc-4bf0-9929-efb9d7fc9035 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:26.712994 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93739fb0-9ecc-4bf0-9929-efb9d7fc9035 service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:26.721189 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-8ddccd39-1a12-4d1e-8b03-5f2beaa28e24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c0195da1ae344f3abc909d9e27f5386', 'request_id': 'req-8ddccd39-1a12-4d1e-8b03-5f2beaa28e24'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:26.722433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1037/2033] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:26 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:10:26.727955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-8ddccd39-1a12-4d1e-8b03-5f2beaa28e24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c0195da1ae344f3abc909d9e27f5386', 'request_id': 'req-8ddccd39-1a12-4d1e-8b03-5f2beaa28e24'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:26.732662 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:26.733867 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-8ddccd39-1a12-4d1e-8b03-5f2beaa28e24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c0195da1ae344f3abc909d9e27f5386', 'request_id': 'req-8ddccd39-1a12-4d1e-8b03-5f2beaa28e24'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:26.747858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-8ddccd39-1a12-4d1e-8b03-5f2beaa28e24 admin admin] The token cache is being invalidate because project 5c0195da1ae344f3abc909d9e27f5386 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:10:26.748143 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-8ddccd39-1a12-4d1e-8b03-5f2beaa28e24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:26.748786 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-8ddccd39-1a12-4d1e-8b03-5f2beaa28e24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:26.750310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 997/2034] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:10:26 2022] DELETE /identity/v3/projects/5c0195da1ae344f3abc909d9e27f5386 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:10:26.754304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93739fb0-9ecc-4bf0-9929-efb9d7fc9035 service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:26.763203 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:26.767881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93739fb0-9ecc-4bf0-9929-efb9d7fc9035 service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:26.808914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddccd39-1a12-4d1e-8b03-5f2beaa28e24 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:26.810368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-8ddccd39-1a12-4d1e-8b03-5f2beaa28e24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:26.831201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fc542-8a90-4b5a-bd9c-a824fec10d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:26.831963 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fc542-8a90-4b5a-bd9c-a824fec10d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:26.832247 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fc542-8a90-4b5a-bd9c-a824fec10d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:26.832521 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03fc542-8a90-4b5a-bd9c-a824fec10d13 admin admin] PATH_INFO: `/v3/users/7ee85b841254413b869acf7de66563ab` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:26.833969 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03fc542-8a90-4b5a-bd9c-a824fec10d13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ee85b841254413b869acf7de66563ab)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:26.836645 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03fc542-8a90-4b5a-bd9c-a824fec10d13 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:26.873451 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddccd39-1a12-4d1e-8b03-5f2beaa28e24 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:26.878275 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1d29b-c197-401f-b3af-a6ff769b4bb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:26.878983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1d29b-c197-401f-b3af-a6ff769b4bb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:26.879365 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1d29b-c197-401f-b3af-a6ff769b4bb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:26.879736 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1d29b-c197-401f-b3af-a6ff769b4bb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:26.881983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-99f1d29b-c197-401f-b3af-a6ff769b4bb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:26.895212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d03fc542-8a90-4b5a-bd9c-a824fec10d13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ee85b841254413b869acf7de66563ab'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:26.903180 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d03fc542-8a90-4b5a-bd9c-a824fec10d13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ee85b841254413b869acf7de66563ab'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:26.911956 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d03fc542-8a90-4b5a-bd9c-a824fec10d13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ee85b841254413b869acf7de66563ab'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:26.912720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d03fc542-8a90-4b5a-bd9c-a824fec10d13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ee85b841254413b869acf7de66563ab'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:26.913430 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d03fc542-8a90-4b5a-bd9c-a824fec10d13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ee85b841254413b869acf7de66563ab'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:26.934158 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1038/2035] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:10:26 2022] DELETE /identity/v3/users/7ee85b841254413b869acf7de66563ab => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:26.953453 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1d29b-c197-401f-b3af-a6ff769b4bb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:26.955125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1d29b-c197-401f-b3af-a6ff769b4bb7 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:26.967512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 998/2036] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:10:27.465400 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fc542-8a90-4b5a-bd9c-a824fec10d13 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:27.467724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-d03fc542-8a90-4b5a-bd9c-a824fec10d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:27.566009 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03fc542-8a90-4b5a-bd9c-a824fec10d13 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:27.570918 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89c785-ffbf-4808-a1ed-f4efda46c5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:27.571577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89c785-ffbf-4808-a1ed-f4efda46c5e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:27.571906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89c785-ffbf-4808-a1ed-f4efda46c5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:27.572180 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89c785-ffbf-4808-a1ed-f4efda46c5e2 admin admin] PATH_INFO: `/v3/projects/e1a4a63a68ea4ca097748f376b77dc03` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:27.573659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89c785-ffbf-4808-a1ed-f4efda46c5e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1a4a63a68ea4ca097748f376b77dc03)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:27.575605 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89c785-ffbf-4808-a1ed-f4efda46c5e2 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:27.601009 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:27.627357 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ec89c785-ffbf-4808-a1ed-f4efda46c5e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1a4a63a68ea4ca097748f376b77dc03', 'request_id': 'req-ec89c785-ffbf-4808-a1ed-f4efda46c5e2'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:27.640176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ec89c785-ffbf-4808-a1ed-f4efda46c5e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1a4a63a68ea4ca097748f376b77dc03', 'request_id': 'req-ec89c785-ffbf-4808-a1ed-f4efda46c5e2'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:27.647954 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ec89c785-ffbf-4808-a1ed-f4efda46c5e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1a4a63a68ea4ca097748f376b77dc03', 'request_id': 'req-ec89c785-ffbf-4808-a1ed-f4efda46c5e2'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:27.662892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ec89c785-ffbf-4808-a1ed-f4efda46c5e2 admin admin] The token cache is being invalidate because project e1a4a63a68ea4ca097748f376b77dc03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:10:27.663215 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ec89c785-ffbf-4808-a1ed-f4efda46c5e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:27.663874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ec89c785-ffbf-4808-a1ed-f4efda46c5e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:27.666264 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1039/2037] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:10:27 2022] DELETE /identity/v3/projects/e1a4a63a68ea4ca097748f376b77dc03 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:27.670915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33da9e-d0b8-4bf2-acf0-ec2faf575dc1 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:27.671258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33da9e-d0b8-4bf2-acf0-ec2faf575dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:27.671540 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33da9e-d0b8-4bf2-acf0-ec2faf575dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:27.693157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-9d33da9e-d0b8-4bf2-acf0-ec2faf575dc1 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:27.695175 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-9d33da9e-d0b8-4bf2-acf0-ec2faf575dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:27.783609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 999/2038] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:27.787300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16281a4-55ff-4cf4-82f9-47ed491c2186 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:27.787612 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16281a4-55ff-4cf4-82f9-47ed491c2186 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:27.787894 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16281a4-55ff-4cf4-82f9-47ed491c2186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:27.812333 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-c16281a4-55ff-4cf4-82f9-47ed491c2186 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:27.813653 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c16281a4-55ff-4cf4-82f9-47ed491c2186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:27.821956 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1040/2039] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:27.826730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33da9e-d0b8-4bf2-acf0-ec2faf575dc1 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:27.836281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33da9e-d0b8-4bf2-acf0-ec2faf575dc1 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:27.840906 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d499ed-8797-4534-b716-41b8cd8c5e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:27.841510 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d499ed-8797-4534-b716-41b8cd8c5e2b admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:27.841793 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d499ed-8797-4534-b716-41b8cd8c5e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:27.842068 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d499ed-8797-4534-b716-41b8cd8c5e2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:27.842712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d499ed-8797-4534-b716-41b8cd8c5e2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:10:27.843016 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d499ed-8797-4534-b716-41b8cd8c5e2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:27.845254 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d499ed-8797-4534-b716-41b8cd8c5e2b admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:27.861914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1000/2040] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:10:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:10:27.863138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16281a4-55ff-4cf4-82f9-47ed491c2186 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:27.872757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16281a4-55ff-4cf4-82f9-47ed491c2186 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:27.877554 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6aa837-1675-4a16-842a-d42b9034f106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:27.878157 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6aa837-1675-4a16-842a-d42b9034f106 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:27.878449 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6aa837-1675-4a16-842a-d42b9034f106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:27.878709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6aa837-1675-4a16-842a-d42b9034f106 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:27.879547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6aa837-1675-4a16-842a-d42b9034f106 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:27.881487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6aa837-1675-4a16-842a-d42b9034f106 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:27.910987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1041/2041] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:10:27 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:27.914892 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d499ed-8797-4534-b716-41b8cd8c5e2b admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:27.926558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d499ed-8797-4534-b716-41b8cd8c5e2b admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:27.960570 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:27.992915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb345b-89f1-499c-a8b1-46f2bc41d9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:27.993570 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb345b-89f1-499c-a8b1-46f2bc41d9c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:27.993816 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb345b-89f1-499c-a8b1-46f2bc41d9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:27.994204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb345b-89f1-499c-a8b1-46f2bc41d9c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:27.994717 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb345b-89f1-499c-a8b1-46f2bc41d9c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:27.996665 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb345b-89f1-499c-a8b1-46f2bc41d9c8 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:28.019271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1001/2042] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:10:27 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:28.023257 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6aa837-1675-4a16-842a-d42b9034f106 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:28.033915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6aa837-1675-4a16-842a-d42b9034f106 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:28.038546 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd7c2b-a276-469b-932a-3230ca1c14a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:28.039178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd7c2b-a276-469b-932a-3230ca1c14a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:28.039462 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd7c2b-a276-469b-932a-3230ca1c14a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:28.039735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd7c2b-a276-469b-932a-3230ca1c14a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:28.040374 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd7c2b-a276-469b-932a-3230ca1c14a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:28.041990 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd7c2b-a276-469b-932a-3230ca1c14a3 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:28.052880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1042/2043] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:28.057335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb345b-89f1-499c-a8b1-46f2bc41d9c8 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:28.068554 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb345b-89f1-499c-a8b1-46f2bc41d9c8 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:28.073616 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27825724-26e1-4611-a1bd-a6b04aa6575a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:28.074218 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27825724-26e1-4611-a1bd-a6b04aa6575a admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:28.074495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27825724-26e1-4611-a1bd-a6b04aa6575a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:28.074764 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27825724-26e1-4611-a1bd-a6b04aa6575a admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:28.075429 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27825724-26e1-4611-a1bd-a6b04aa6575a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:28.077004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27825724-26e1-4611-a1bd-a6b04aa6575a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:28.087371 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1002/2044] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:28.091605 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd7c2b-a276-469b-932a-3230ca1c14a3 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:28.101863 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd7c2b-a276-469b-932a-3230ca1c14a3 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:28.106877 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d0924-0db0-484e-87e3-7642a69a5530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:28.107658 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d0924-0db0-484e-87e3-7642a69a5530 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:28.108071 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d0924-0db0-484e-87e3-7642a69a5530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:28.108418 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d0924-0db0-484e-87e3-7642a69a5530 admin admin] PATH_INFO: `/v3/projects/de8f94d275e3408385a970b591285206/users/df0a88c35f8d468fb60e90141f8c1b17/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:28.125423 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d0924-0db0-484e-87e3-7642a69a5530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de8f94d275e3408385a970b591285206, user_id=df0a88c35f8d468fb60e90141f8c1b17, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:28.128628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d0924-0db0-484e-87e3-7642a69a5530 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:28.161423 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1043/2045] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:28 2022] PUT /identity/v3/projects/de8f94d275e3408385a970b591285206/users/df0a88c35f8d468fb60e90141f8c1b17/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:28.163793 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b356ab0-74ef-4cb2-9b68-daf786e1f065 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:28.164378 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b356ab0-74ef-4cb2-9b68-daf786e1f065 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:28.164748 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b356ab0-74ef-4cb2-9b68-daf786e1f065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:28.208391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-5b356ab0-74ef-4cb2-9b68-daf786e1f065 None None] MFA Rules not processed for user `df0a88c35f8d468fb60e90141f8c1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:28.210218 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-5b356ab0-74ef-4cb2-9b68-daf786e1f065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:28.287088 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1003/2046] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:28.291508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d0924-0db0-484e-87e3-7642a69a5530 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:28.390126 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d0924-0db0-484e-87e3-7642a69a5530 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:28.397015 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2259cb5-3000-4836-ad9b-cdb7a5a59808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:28.397628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2259cb5-3000-4836-ad9b-cdb7a5a59808 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:28.397911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2259cb5-3000-4836-ad9b-cdb7a5a59808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:28.398217 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2259cb5-3000-4836-ad9b-cdb7a5a59808 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:28.398910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2259cb5-3000-4836-ad9b-cdb7a5a59808 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:28.400883 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2259cb5-3000-4836-ad9b-cdb7a5a59808 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:28.433083 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1044/2047] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:10:28 2022] POST /identity/v3/projects => generated 387 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:28.437921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b356ab0-74ef-4cb2-9b68-daf786e1f065 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:28.451537 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b356ab0-74ef-4cb2-9b68-daf786e1f065 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:28.522276 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712232f-c4a2-4e6f-8e2f-f0db39ce28d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:28.523254 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712232f-c4a2-4e6f-8e2f-f0db39ce28d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:28.525568 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712232f-c4a2-4e6f-8e2f-f0db39ce28d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:28.525862 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712232f-c4a2-4e6f-8e2f-f0db39ce28d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:28.526911 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712232f-c4a2-4e6f-8e2f-f0db39ce28d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:28.528476 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712232f-c4a2-4e6f-8e2f-f0db39ce28d7 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:28.574460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1004/2048] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:10:28 2022] POST /identity/v3/users => generated 300 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:28.579057 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2259cb5-3000-4836-ad9b-cdb7a5a59808 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:28.589953 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2259cb5-3000-4836-ad9b-cdb7a5a59808 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:28.596242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2cc21-d9c1-4bdd-ae3c-80fb7cffc133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:28.596857 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2cc21-d9c1-4bdd-ae3c-80fb7cffc133 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:28.597142 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2cc21-d9c1-4bdd-ae3c-80fb7cffc133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:28.597414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec2cc21-d9c1-4bdd-ae3c-80fb7cffc133 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:28.598390 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec2cc21-d9c1-4bdd-ae3c-80fb7cffc133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:28.600037 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec2cc21-d9c1-4bdd-ae3c-80fb7cffc133 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:28.617261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1045/2049] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:28.618364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712232f-c4a2-4e6f-8e2f-f0db39ce28d7 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:28.638909 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712232f-c4a2-4e6f-8e2f-f0db39ce28d7 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:28.648871 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeafd1f-513e-45f8-89ee-5722f9ed3732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:28.649564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeafd1f-513e-45f8-89ee-5722f9ed3732 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:28.649849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeafd1f-513e-45f8-89ee-5722f9ed3732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:28.650126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeafd1f-513e-45f8-89ee-5722f9ed3732 admin admin] PATH_INFO: `/v3/projects/27d331b4c5dd49e18cad3e9a03710b9a/users/94d062b4a41a4b9cb7457ecd79494ba0/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:28.675203 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeafd1f-513e-45f8-89ee-5722f9ed3732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27d331b4c5dd49e18cad3e9a03710b9a, user_id=94d062b4a41a4b9cb7457ecd79494ba0, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:28.678350 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeafd1f-513e-45f8-89ee-5722f9ed3732 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:28.692874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1005/2050] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:28 2022] PUT /identity/v3/projects/27d331b4c5dd49e18cad3e9a03710b9a/users/94d062b4a41a4b9cb7457ecd79494ba0/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:28.696845 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2cc21-d9c1-4bdd-ae3c-80fb7cffc133 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:28.707643 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec2cc21-d9c1-4bdd-ae3c-80fb7cffc133 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:28.727974 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:28.780531 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a58f454-de23-439e-b1fb-d1a2bd3b7864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:28.781278 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58f454-de23-439e-b1fb-d1a2bd3b7864 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:28.781558 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58f454-de23-439e-b1fb-d1a2bd3b7864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:28.781887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58f454-de23-439e-b1fb-d1a2bd3b7864 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:28.783009 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a58f454-de23-439e-b1fb-d1a2bd3b7864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:28.784951 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a58f454-de23-439e-b1fb-d1a2bd3b7864 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:28.801249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1046/2051] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:10:28.802556 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeafd1f-513e-45f8-89ee-5722f9ed3732 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:28.810549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:28.815728 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeafd1f-513e-45f8-89ee-5722f9ed3732 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:28.824584 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce34e73-5e63-4344-816d-3ac5741dd20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:28.825380 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce34e73-5e63-4344-816d-3ac5741dd20d admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:28.825757 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce34e73-5e63-4344-816d-3ac5741dd20d admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:28.826132 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce34e73-5e63-4344-816d-3ac5741dd20d admin admin] PATH_INFO: `/v3/projects/27d331b4c5dd49e18cad3e9a03710b9a/users/94d062b4a41a4b9cb7457ecd79494ba0/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:28.832532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce34e73-5e63-4344-816d-3ac5741dd20d admin admin] RBAC: Authorizing `identity:create_grant(project_id=27d331b4c5dd49e18cad3e9a03710b9a, user_id=94d062b4a41a4b9cb7457ecd79494ba0, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:28.835245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce34e73-5e63-4344-816d-3ac5741dd20d admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:28.848255 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1006/2052] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:28 2022] PUT /identity/v3/projects/27d331b4c5dd49e18cad3e9a03710b9a/users/94d062b4a41a4b9cb7457ecd79494ba0/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:28.859084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa051a-4e18-48bd-a4b4-ede1a8e760f1 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:28.859395 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa051a-4e18-48bd-a4b4-ede1a8e760f1 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:28.859692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa051a-4e18-48bd-a4b4-ede1a8e760f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:28.904379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-40fa051a-4e18-48bd-a4b4-ede1a8e760f1 None None] MFA Rules not processed for user `94d062b4a41a4b9cb7457ecd79494ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:28.913193 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-40fa051a-4e18-48bd-a4b4-ede1a8e760f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:29.005224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1047/2053] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:28 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:29.016035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce34e73-5e63-4344-816d-3ac5741dd20d admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:29.018453 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-8ce34e73-5e63-4344-816d-3ac5741dd20d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:29.077082 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce34e73-5e63-4344-816d-3ac5741dd20d admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:29.081478 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e774372-8510-4809-9373-0f2bada2cb51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:29.081913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e774372-8510-4809-9373-0f2bada2cb51 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:29.082198 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e774372-8510-4809-9373-0f2bada2cb51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:29.082469 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e774372-8510-4809-9373-0f2bada2cb51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:29.091791 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e774372-8510-4809-9373-0f2bada2cb51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:29.093257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e774372-8510-4809-9373-0f2bada2cb51 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:29.105637 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1007/2054] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:10:29.679761 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa051a-4e18-48bd-a4b4-ede1a8e760f1 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:29.747890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa051a-4e18-48bd-a4b4-ede1a8e760f1 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:29.752915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34bf56-d47b-4b83-bd4f-11ecc7838251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:29.753538 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34bf56-d47b-4b83-bd4f-11ecc7838251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:29.753815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34bf56-d47b-4b83-bd4f-11ecc7838251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:29.754086 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34bf56-d47b-4b83-bd4f-11ecc7838251 admin admin] PATH_INFO: `/v3/users/df0a88c35f8d468fb60e90141f8c1b17` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:29.755251 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34bf56-d47b-4b83-bd4f-11ecc7838251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df0a88c35f8d468fb60e90141f8c1b17)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:29.757155 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce34bf56-d47b-4b83-bd4f-11ecc7838251 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:29.802317 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:29.804337 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ce34bf56-d47b-4b83-bd4f-11ecc7838251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df0a88c35f8d468fb60e90141f8c1b17'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:29.812491 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ce34bf56-d47b-4b83-bd4f-11ecc7838251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df0a88c35f8d468fb60e90141f8c1b17'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:29.820548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ce34bf56-d47b-4b83-bd4f-11ecc7838251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df0a88c35f8d468fb60e90141f8c1b17'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:29.821206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ce34bf56-d47b-4b83-bd4f-11ecc7838251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df0a88c35f8d468fb60e90141f8c1b17'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:29.821860 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ce34bf56-d47b-4b83-bd4f-11ecc7838251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df0a88c35f8d468fb60e90141f8c1b17'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:29.843624 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1048/2055] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:10:29 2022] DELETE /identity/v3/users/df0a88c35f8d468fb60e90141f8c1b17 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:29.848695 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe91505-0c8c-4f9c-8b4c-8ef4d497513b None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:29.849016 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe91505-0c8c-4f9c-8b4c-8ef4d497513b None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:29.849293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe91505-0c8c-4f9c-8b4c-8ef4d497513b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:29.867730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-efe91505-0c8c-4f9c-8b4c-8ef4d497513b None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:29.868926 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-efe91505-0c8c-4f9c-8b4c-8ef4d497513b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:29.928185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1008/2056] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:29.940852 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34bf56-d47b-4b83-bd4f-11ecc7838251 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:29.942569 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-ce34bf56-d47b-4b83-bd4f-11ecc7838251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:30.012717 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34bf56-d47b-4b83-bd4f-11ecc7838251 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:30.017230 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4baa5b3-05a2-4b0d-8a5e-d495e6c1f4f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:30.018093 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baa5b3-05a2-4b0d-8a5e-d495e6c1f4f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:30.018344 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baa5b3-05a2-4b0d-8a5e-d495e6c1f4f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:30.018617 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baa5b3-05a2-4b0d-8a5e-d495e6c1f4f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:30.028312 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4baa5b3-05a2-4b0d-8a5e-d495e6c1f4f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:30.029718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4baa5b3-05a2-4b0d-8a5e-d495e6c1f4f3 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:30.055603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1049/2057] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:10:30.393002 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe91505-0c8c-4f9c-8b4c-8ef4d497513b None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:30.394503 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-efe91505-0c8c-4f9c-8b4c-8ef4d497513b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:30.406356 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe91505-0c8c-4f9c-8b4c-8ef4d497513b None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:30.410954 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a4666-9e65-4810-a3ca-545e18dc7205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:30.411614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a4666-9e65-4810-a3ca-545e18dc7205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:30.411896 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a4666-9e65-4810-a3ca-545e18dc7205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:30.412167 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a4666-9e65-4810-a3ca-545e18dc7205 admin admin] PATH_INFO: `/v3/projects/de8f94d275e3408385a970b591285206` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:30.413497 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a4666-9e65-4810-a3ca-545e18dc7205 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de8f94d275e3408385a970b591285206)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:30.415488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a4666-9e65-4810-a3ca-545e18dc7205 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:30.468701 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-3e3a4666-9e65-4810-a3ca-545e18dc7205 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de8f94d275e3408385a970b591285206', 'request_id': 'req-3e3a4666-9e65-4810-a3ca-545e18dc7205'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:30.474592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-3e3a4666-9e65-4810-a3ca-545e18dc7205 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de8f94d275e3408385a970b591285206', 'request_id': 'req-3e3a4666-9e65-4810-a3ca-545e18dc7205'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:30.479922 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-3e3a4666-9e65-4810-a3ca-545e18dc7205 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de8f94d275e3408385a970b591285206', 'request_id': 'req-3e3a4666-9e65-4810-a3ca-545e18dc7205'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:30.492482 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-3e3a4666-9e65-4810-a3ca-545e18dc7205 admin admin] The token cache is being invalidate because project de8f94d275e3408385a970b591285206 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:10:30.492760 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-3e3a4666-9e65-4810-a3ca-545e18dc7205 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:30.493397 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-3e3a4666-9e65-4810-a3ca-545e18dc7205 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:30.494849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1009/2058] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:10:30 2022] DELETE /identity/v3/projects/de8f94d275e3408385a970b591285206 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:30.498771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4baa5b3-05a2-4b0d-8a5e-d495e6c1f4f3 service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:30.509038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4baa5b3-05a2-4b0d-8a5e-d495e6c1f4f3 service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:30.564509 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62aaef-ab04-4e2b-97ed-5a0eda7e1cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:30.565133 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62aaef-ab04-4e2b-97ed-5a0eda7e1cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:30.565415 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62aaef-ab04-4e2b-97ed-5a0eda7e1cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:30.565677 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62aaef-ab04-4e2b-97ed-5a0eda7e1cf2 admin admin] PATH_INFO: `/v3/users/94d062b4a41a4b9cb7457ecd79494ba0` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:30.566870 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62aaef-ab04-4e2b-97ed-5a0eda7e1cf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94d062b4a41a4b9cb7457ecd79494ba0)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:30.568720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62aaef-ab04-4e2b-97ed-5a0eda7e1cf2 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:30.616702 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-7a62aaef-ab04-4e2b-97ed-5a0eda7e1cf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94d062b4a41a4b9cb7457ecd79494ba0'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:30.623571 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-7a62aaef-ab04-4e2b-97ed-5a0eda7e1cf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94d062b4a41a4b9cb7457ecd79494ba0'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:30.632263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-7a62aaef-ab04-4e2b-97ed-5a0eda7e1cf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94d062b4a41a4b9cb7457ecd79494ba0'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:30.633042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-7a62aaef-ab04-4e2b-97ed-5a0eda7e1cf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94d062b4a41a4b9cb7457ecd79494ba0'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:30.633646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-7a62aaef-ab04-4e2b-97ed-5a0eda7e1cf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94d062b4a41a4b9cb7457ecd79494ba0'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:30.638219 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:30.653766 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1050/2059] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:10:30 2022] DELETE /identity/v3/users/94d062b4a41a4b9cb7457ecd79494ba0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:30.919225 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a4666-9e65-4810-a3ca-545e18dc7205 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:30.920648 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-3e3a4666-9e65-4810-a3ca-545e18dc7205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:30.935017 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a4666-9e65-4810-a3ca-545e18dc7205 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:30.949181 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:30.993717 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85b594-0f41-4e43-81b5-5bfbb01d2c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:30.994356 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85b594-0f41-4e43-81b5-5bfbb01d2c64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:30.994635 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85b594-0f41-4e43-81b5-5bfbb01d2c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:30.994987 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85b594-0f41-4e43-81b5-5bfbb01d2c64 admin admin] PATH_INFO: `/v3/projects/27d331b4c5dd49e18cad3e9a03710b9a` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:30.996148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85b594-0f41-4e43-81b5-5bfbb01d2c64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27d331b4c5dd49e18cad3e9a03710b9a)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:30.998026 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85b594-0f41-4e43-81b5-5bfbb01d2c64 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:31.042082 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4d85b594-0f41-4e43-81b5-5bfbb01d2c64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27d331b4c5dd49e18cad3e9a03710b9a', 'request_id': 'req-4d85b594-0f41-4e43-81b5-5bfbb01d2c64'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:31.048601 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4d85b594-0f41-4e43-81b5-5bfbb01d2c64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27d331b4c5dd49e18cad3e9a03710b9a', 'request_id': 'req-4d85b594-0f41-4e43-81b5-5bfbb01d2c64'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:31.054026 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4d85b594-0f41-4e43-81b5-5bfbb01d2c64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27d331b4c5dd49e18cad3e9a03710b9a', 'request_id': 'req-4d85b594-0f41-4e43-81b5-5bfbb01d2c64'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:31.067611 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4d85b594-0f41-4e43-81b5-5bfbb01d2c64 admin admin] The token cache is being invalidate because project 27d331b4c5dd49e18cad3e9a03710b9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:10:31.067902 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4d85b594-0f41-4e43-81b5-5bfbb01d2c64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:31.068532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4d85b594-0f41-4e43-81b5-5bfbb01d2c64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:31.070004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1010/2060] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:10:30 2022] DELETE /identity/v3/projects/27d331b4c5dd49e18cad3e9a03710b9a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:31.075203 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff3f79-ee13-48ce-89a5-cb9917111159 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:31.075657 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff3f79-ee13-48ce-89a5-cb9917111159 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:31.075765 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff3f79-ee13-48ce-89a5-cb9917111159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:31.095164 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-70ff3f79-ee13-48ce-89a5-cb9917111159 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:31.096350 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-70ff3f79-ee13-48ce-89a5-cb9917111159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:31.155575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1051/2061] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:31.163116 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61714adb-6758-402f-9af0-7e7a0f743127 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:31.163500 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61714adb-6758-402f-9af0-7e7a0f743127 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:31.163893 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61714adb-6758-402f-9af0-7e7a0f743127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:31.185036 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-61714adb-6758-402f-9af0-7e7a0f743127 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:31.186356 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-61714adb-6758-402f-9af0-7e7a0f743127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:31.195611 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1011/2062] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:10:31.200089 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff3f79-ee13-48ce-89a5-cb9917111159 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:31.209775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff3f79-ee13-48ce-89a5-cb9917111159 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:31.214565 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf20f39-b173-4cf8-aebc-229c01953b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:31.215279 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf20f39-b173-4cf8-aebc-229c01953b18 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:31.215548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf20f39-b173-4cf8-aebc-229c01953b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:31.215821 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf20f39-b173-4cf8-aebc-229c01953b18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:31.216460 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf20f39-b173-4cf8-aebc-229c01953b18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:10:31.216759 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf20f39-b173-4cf8-aebc-229c01953b18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:31.218356 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf20f39-b173-4cf8-aebc-229c01953b18 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:31.230105 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1052/2063] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:10:31.234626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61714adb-6758-402f-9af0-7e7a0f743127 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:31.244609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61714adb-6758-402f-9af0-7e7a0f743127 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:31.249702 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336ab78-5a57-40a0-9772-00964d79bd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:31.250312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336ab78-5a57-40a0-9772-00964d79bd54 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:31.250637 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336ab78-5a57-40a0-9772-00964d79bd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:31.251442 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336ab78-5a57-40a0-9772-00964d79bd54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:31.252089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336ab78-5a57-40a0-9772-00964d79bd54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:31.254000 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336ab78-5a57-40a0-9772-00964d79bd54 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:31.280715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1012/2064] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:10:31 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:10:31.286687 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf20f39-b173-4cf8-aebc-229c01953b18 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:31.297199 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf20f39-b173-4cf8-aebc-229c01953b18 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:31.353268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e777006-542c-4714-86e5-f7d9d1f4a0e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:31.353872 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e777006-542c-4714-86e5-f7d9d1f4a0e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:31.354143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e777006-542c-4714-86e5-f7d9d1f4a0e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:31.354411 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e777006-542c-4714-86e5-f7d9d1f4a0e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:31.355099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e777006-542c-4714-86e5-f7d9d1f4a0e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:31.356992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e777006-542c-4714-86e5-f7d9d1f4a0e0 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:31.380972 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1053/2065] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:10:31 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:31.385153 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336ab78-5a57-40a0-9772-00964d79bd54 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:31.396059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336ab78-5a57-40a0-9772-00964d79bd54 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:31.400857 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a84342-63c5-471c-a8ff-623809c254ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:31.401463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a84342-63c5-471c-a8ff-623809c254ec admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:31.401738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a84342-63c5-471c-a8ff-623809c254ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:31.402005 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a84342-63c5-471c-a8ff-623809c254ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:31.402679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a84342-63c5-471c-a8ff-623809c254ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:31.404315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a84342-63c5-471c-a8ff-623809c254ec admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:31.417876 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1013/2066] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:31.421985 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e777006-542c-4714-86e5-f7d9d1f4a0e0 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:31.438471 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e777006-542c-4714-86e5-f7d9d1f4a0e0 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:31.443986 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d10e55-7374-4590-9552-9dfd172754da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:31.444582 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d10e55-7374-4590-9552-9dfd172754da admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:31.444886 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d10e55-7374-4590-9552-9dfd172754da admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:31.445155 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d10e55-7374-4590-9552-9dfd172754da admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:31.445790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d10e55-7374-4590-9552-9dfd172754da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:31.447416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d10e55-7374-4590-9552-9dfd172754da admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:31.458312 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1054/2067] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:31.462532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a84342-63c5-471c-a8ff-623809c254ec admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:31.473115 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a84342-63c5-471c-a8ff-623809c254ec admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:31.477741 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464fa01-c3cc-4da4-a045-07fa2902570b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:31.478382 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464fa01-c3cc-4da4-a045-07fa2902570b admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:31.478660 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464fa01-c3cc-4da4-a045-07fa2902570b admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:31.478933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464fa01-c3cc-4da4-a045-07fa2902570b admin admin] PATH_INFO: `/v3/projects/198bf26f4a924b268ece1bc5885bf838/users/cde19b5db1ea459499f8c5e7db4ac102/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:31.493955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464fa01-c3cc-4da4-a045-07fa2902570b admin admin] RBAC: Authorizing `identity:create_grant(project_id=198bf26f4a924b268ece1bc5885bf838, user_id=cde19b5db1ea459499f8c5e7db4ac102, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:31.496665 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464fa01-c3cc-4da4-a045-07fa2902570b admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:31.509898 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1014/2068] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:31 2022] PUT /identity/v3/projects/198bf26f4a924b268ece1bc5885bf838/users/cde19b5db1ea459499f8c5e7db4ac102/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:31.519891 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcdbbb-3412-4d51-85bb-8198d0aa0bec None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:31.520176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcdbbb-3412-4d51-85bb-8198d0aa0bec None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:31.520450 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcdbbb-3412-4d51-85bb-8198d0aa0bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:31.548643 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-2bdcdbbb-3412-4d51-85bb-8198d0aa0bec None None] MFA Rules not processed for user `cde19b5db1ea459499f8c5e7db4ac102`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:31.549867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-2bdcdbbb-3412-4d51-85bb-8198d0aa0bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:31.588484 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:31.614953 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1055/2069] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:31.619537 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464fa01-c3cc-4da4-a045-07fa2902570b admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:31.664051 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:31.681481 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464fa01-c3cc-4da4-a045-07fa2902570b admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:31.686228 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c81a9-686c-4777-8b19-83a6587669a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:31.686820 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c81a9-686c-4777-8b19-83a6587669a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:31.687120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c81a9-686c-4777-8b19-83a6587669a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:31.687392 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c81a9-686c-4777-8b19-83a6587669a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:31.688049 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c81a9-686c-4777-8b19-83a6587669a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:31.689905 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c81a9-686c-4777-8b19-83a6587669a1 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:31.730532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1015/2070] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:10:31 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:31.732111 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcdbbb-3412-4d51-85bb-8198d0aa0bec None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:31.742761 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcdbbb-3412-4d51-85bb-8198d0aa0bec None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:31.798823 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3978c-a130-4606-8a5a-7c9ef616eec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:31.799500 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3978c-a130-4606-8a5a-7c9ef616eec4 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:31.799778 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3978c-a130-4606-8a5a-7c9ef616eec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:31.800050 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3978c-a130-4606-8a5a-7c9ef616eec4 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:31.800681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a3978c-a130-4606-8a5a-7c9ef616eec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:31.802610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a3978c-a130-4606-8a5a-7c9ef616eec4 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:31.825430 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1056/2071] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:10:31 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:31.829986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c81a9-686c-4777-8b19-83a6587669a1 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:31.840682 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c81a9-686c-4777-8b19-83a6587669a1 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:31.846603 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d4d44-4a4c-4b60-8fe9-9daad33a894a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:31.847238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d4d44-4a4c-4b60-8fe9-9daad33a894a admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:31.847541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d4d44-4a4c-4b60-8fe9-9daad33a894a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:31.848249 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d4d44-4a4c-4b60-8fe9-9daad33a894a admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:31.848479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d4d44-4a4c-4b60-8fe9-9daad33a894a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:31.850227 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d4d44-4a4c-4b60-8fe9-9daad33a894a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:31.862141 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1016/2072] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:31.866118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3978c-a130-4606-8a5a-7c9ef616eec4 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:31.876371 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3978c-a130-4606-8a5a-7c9ef616eec4 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:31.881439 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998e4ed-06bd-4cfc-bf52-858e4ac1c846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:31.882036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998e4ed-06bd-4cfc-bf52-858e4ac1c846 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:31.882334 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998e4ed-06bd-4cfc-bf52-858e4ac1c846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:31.882605 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998e4ed-06bd-4cfc-bf52-858e4ac1c846 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:31.883274 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998e4ed-06bd-4cfc-bf52-858e4ac1c846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:31.884888 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998e4ed-06bd-4cfc-bf52-858e4ac1c846 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:31.898722 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1057/2073] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:31.900205 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d4d44-4a4c-4b60-8fe9-9daad33a894a admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:31.910968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d4d44-4a4c-4b60-8fe9-9daad33a894a admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:31.915704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c020bd-52ac-4c46-ae9e-a619a466c2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:31.916356 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c020bd-52ac-4c46-ae9e-a619a466c2d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:31.916635 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c020bd-52ac-4c46-ae9e-a619a466c2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:31.916908 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c020bd-52ac-4c46-ae9e-a619a466c2d5 admin admin] PATH_INFO: `/v3/projects/a3d771ee834f469681d3da1eb2478139/users/0c19928346154e27bf2e8e97361b32ac/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:31.931287 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c020bd-52ac-4c46-ae9e-a619a466c2d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3d771ee834f469681d3da1eb2478139, user_id=0c19928346154e27bf2e8e97361b32ac, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:31.934045 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c020bd-52ac-4c46-ae9e-a619a466c2d5 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:31.946635 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1017/2074] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:31 2022] PUT /identity/v3/projects/a3d771ee834f469681d3da1eb2478139/users/0c19928346154e27bf2e8e97361b32ac/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:10:31.957782 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851cd2d-fe4d-4d32-b871-6369c9a18760 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:31.958073 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851cd2d-fe4d-4d32-b871-6369c9a18760 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:31.958347 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851cd2d-fe4d-4d32-b871-6369c9a18760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:31.986671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-4851cd2d-fe4d-4d32-b871-6369c9a18760 None None] MFA Rules not processed for user `0c19928346154e27bf2e8e97361b32ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:31.987934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-4851cd2d-fe4d-4d32-b871-6369c9a18760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:32.051419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1058/2075] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:32.056174 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c020bd-52ac-4c46-ae9e-a619a466c2d5 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:32.116573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c020bd-52ac-4c46-ae9e-a619a466c2d5 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:32.121209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8f6ab-8e9a-48de-8bee-bafa10628d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:32.121808 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8f6ab-8e9a-48de-8bee-bafa10628d73 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:32.122091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8f6ab-8e9a-48de-8bee-bafa10628d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:32.122364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8f6ab-8e9a-48de-8bee-bafa10628d73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:32.122994 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8f6ab-8e9a-48de-8bee-bafa10628d73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:32.124920 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8f6ab-8e9a-48de-8bee-bafa10628d73 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:32.151062 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1018/2076] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:10:32 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:32.155331 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851cd2d-fe4d-4d32-b871-6369c9a18760 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:32.165827 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851cd2d-fe4d-4d32-b871-6369c9a18760 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:32.210321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:32.222461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de2582-95f3-44bf-ac1d-2bfcfaf72426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:32.223052 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de2582-95f3-44bf-ac1d-2bfcfaf72426 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:32.223360 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de2582-95f3-44bf-ac1d-2bfcfaf72426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:32.223627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de2582-95f3-44bf-ac1d-2bfcfaf72426 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:32.224234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de2582-95f3-44bf-ac1d-2bfcfaf72426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:32.226139 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97de2582-95f3-44bf-ac1d-2bfcfaf72426 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:32.246222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1059/2077] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:10:32 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:32.250160 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8f6ab-8e9a-48de-8bee-bafa10628d73 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:32.261529 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8f6ab-8e9a-48de-8bee-bafa10628d73 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:32.266958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ca4c3-a8f3-49b4-b435-02dfb176fd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:32.267596 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ca4c3-a8f3-49b4-b435-02dfb176fd12 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:32.267876 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ca4c3-a8f3-49b4-b435-02dfb176fd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:32.268150 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ca4c3-a8f3-49b4-b435-02dfb176fd12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:32.268812 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ca4c3-a8f3-49b4-b435-02dfb176fd12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:32.270428 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ca4c3-a8f3-49b4-b435-02dfb176fd12 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:32.281160 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1019/2078] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:32.285295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de2582-95f3-44bf-ac1d-2bfcfaf72426 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:32.295248 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de2582-95f3-44bf-ac1d-2bfcfaf72426 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:32.299796 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d723f86-9498-4b81-bbe7-3102a251c1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:32.300431 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d723f86-9498-4b81-bbe7-3102a251c1ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:32.300704 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d723f86-9498-4b81-bbe7-3102a251c1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:32.300972 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d723f86-9498-4b81-bbe7-3102a251c1ab admin admin] PATH_INFO: `/v3/projects/3b1529dc077545999f0e454334a4c8eb/users/9574fff7edda41f2b466d6ec519445f9/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:32.315651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d723f86-9498-4b81-bbe7-3102a251c1ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b1529dc077545999f0e454334a4c8eb, user_id=9574fff7edda41f2b466d6ec519445f9, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:32.318286 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d723f86-9498-4b81-bbe7-3102a251c1ab admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:32.333028 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1060/2079] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:32 2022] PUT /identity/v3/projects/3b1529dc077545999f0e454334a4c8eb/users/9574fff7edda41f2b466d6ec519445f9/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:32.336873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ca4c3-a8f3-49b4-b435-02dfb176fd12 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:32.347394 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ca4c3-a8f3-49b4-b435-02dfb176fd12 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:32.405057 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4239e1fe-7b65-4e8c-873d-5c70b1bd3315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:32.405763 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239e1fe-7b65-4e8c-873d-5c70b1bd3315 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:32.405939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239e1fe-7b65-4e8c-873d-5c70b1bd3315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:32.406214 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4239e1fe-7b65-4e8c-873d-5c70b1bd3315 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:32.406916 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239e1fe-7b65-4e8c-873d-5c70b1bd3315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:32.408539 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4239e1fe-7b65-4e8c-873d-5c70b1bd3315 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:32.421168 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1020/2080] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:10:32.425767 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d723f86-9498-4b81-bbe7-3102a251c1ab admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:32.436430 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d723f86-9498-4b81-bbe7-3102a251c1ab admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:32.442684 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c16ed-05e8-496c-af4b-3972f322c5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:32.443453 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c16ed-05e8-496c-af4b-3972f322c5a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:32.443805 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c16ed-05e8-496c-af4b-3972f322c5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:32.444148 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c16ed-05e8-496c-af4b-3972f322c5a8 admin admin] PATH_INFO: `/v3/projects/3b1529dc077545999f0e454334a4c8eb/users/9574fff7edda41f2b466d6ec519445f9/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:32.446519 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c16ed-05e8-496c-af4b-3972f322c5a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b1529dc077545999f0e454334a4c8eb, user_id=9574fff7edda41f2b466d6ec519445f9, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:32.449463 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c16ed-05e8-496c-af4b-3972f322c5a8 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:32.460947 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1061/2081] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:32 2022] PUT /identity/v3/projects/3b1529dc077545999f0e454334a4c8eb/users/9574fff7edda41f2b466d6ec519445f9/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:32.471925 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ab7c4-c7c7-4f74-ad36-54b8a48030f7 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:32.472215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ab7c4-c7c7-4f74-ad36-54b8a48030f7 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:32.472492 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ab7c4-c7c7-4f74-ad36-54b8a48030f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:32.481023 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:32.506606 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-a14ab7c4-c7c7-4f74-ad36-54b8a48030f7 None None] MFA Rules not processed for user `9574fff7edda41f2b466d6ec519445f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:32.515142 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-a14ab7c4-c7c7-4f74-ad36-54b8a48030f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:32.583507 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1021/2082] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:32.595019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c16ed-05e8-496c-af4b-3972f322c5a8 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:32.596819 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-679c16ed-05e8-496c-af4b-3972f322c5a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:32.654268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c16ed-05e8-496c-af4b-3972f322c5a8 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:32.657928 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b75ad8-aad3-4248-9906-734dd2866556 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:32.658519 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b75ad8-aad3-4248-9906-734dd2866556 service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:32.658798 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b75ad8-aad3-4248-9906-734dd2866556 service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:32.659086 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b75ad8-aad3-4248-9906-734dd2866556 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:32.715318 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b75ad8-aad3-4248-9906-734dd2866556 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:32.716683 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b75ad8-aad3-4248-9906-734dd2866556 service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:32.725440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1062/2083] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:10:34.228681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ab7c4-c7c7-4f74-ad36-54b8a48030f7 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:34.238928 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ab7c4-c7c7-4f74-ad36-54b8a48030f7 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:34.242566 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e9985-bd18-484a-b662-5e9c9a188675 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:34.243274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e9985-bd18-484a-b662-5e9c9a188675 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:34.243659 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e9985-bd18-484a-b662-5e9c9a188675 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:34.244320 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e9985-bd18-484a-b662-5e9c9a188675 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:34.301583 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e9985-bd18-484a-b662-5e9c9a188675 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:34.303211 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e9985-bd18-484a-b662-5e9c9a188675 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:34.312103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1022/2084] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:10:34.891029 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b75ad8-aad3-4248-9906-734dd2866556 service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:34.893508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-76b75ad8-aad3-4248-9906-734dd2866556 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:34.905250 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b75ad8-aad3-4248-9906-734dd2866556 service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:34.917222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a14e337-a89b-47bc-99d6-210818609712 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:34.917831 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14e337-a89b-47bc-99d6-210818609712 service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:34.918111 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14e337-a89b-47bc-99d6-210818609712 service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:34.918411 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14e337-a89b-47bc-99d6-210818609712 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:34.928079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14e337-a89b-47bc-99d6-210818609712 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:34.929782 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a14e337-a89b-47bc-99d6-210818609712 service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:34.939293 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1063/2085] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:10:37.597174 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e9985-bd18-484a-b662-5e9c9a188675 service cinder] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:37.598793 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-041e9985-bd18-484a-b662-5e9c9a188675 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:37.675580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e9985-bd18-484a-b662-5e9c9a188675 service cinder] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:37.680396 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a78a13-38e4-4ebe-a184-2e9e334be0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:37.681383 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a78a13-38e4-4ebe-a184-2e9e334be0a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:37.681383 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a78a13-38e4-4ebe-a184-2e9e334be0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:37.681851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a78a13-38e4-4ebe-a184-2e9e334be0a9 admin admin] PATH_INFO: `/v3/users/1a4e5504823747fa96216667129b2107` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:37.682975 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a78a13-38e4-4ebe-a184-2e9e334be0a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a4e5504823747fa96216667129b2107)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:37.685059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a78a13-38e4-4ebe-a184-2e9e334be0a9 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:37.737410 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:37.756625 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-76a78a13-38e4-4ebe-a184-2e9e334be0a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a4e5504823747fa96216667129b2107'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:37.757347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-76a78a13-38e4-4ebe-a184-2e9e334be0a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a4e5504823747fa96216667129b2107'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:37.773878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-76a78a13-38e4-4ebe-a184-2e9e334be0a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a4e5504823747fa96216667129b2107'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:37.792790 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-76a78a13-38e4-4ebe-a184-2e9e334be0a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a4e5504823747fa96216667129b2107'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:37.793585 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-76a78a13-38e4-4ebe-a184-2e9e334be0a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a4e5504823747fa96216667129b2107'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:37.809724 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1023/2086] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:10:37 2022] DELETE /identity/v3/users/1a4e5504823747fa96216667129b2107 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:37.814308 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0bca2-2bfd-47ce-a3bc-07c266a4750d None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:37.814771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0bca2-2bfd-47ce-a3bc-07c266a4750d None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:37.814861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0bca2-2bfd-47ce-a3bc-07c266a4750d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:37.834514 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-c6b0bca2-2bfd-47ce-a3bc-07c266a4750d None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:37.835757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c6b0bca2-2bfd-47ce-a3bc-07c266a4750d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:37.873936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:37.897369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1064/2087] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:37.912259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a78a13-38e4-4ebe-a184-2e9e334be0a9 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:37.913789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-76a78a13-38e4-4ebe-a184-2e9e334be0a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:37.979651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a78a13-38e4-4ebe-a184-2e9e334be0a9 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:37.985177 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75837596-506d-4d46-9736-12ec885afdd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:37.986229 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75837596-506d-4d46-9736-12ec885afdd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:37.986547 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75837596-506d-4d46-9736-12ec885afdd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:37.986964 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75837596-506d-4d46-9736-12ec885afdd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:37.997633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75837596-506d-4d46-9736-12ec885afdd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:37.999093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75837596-506d-4d46-9736-12ec885afdd0 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:38.009091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1024/2088] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:10:38.281372 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0bca2-2bfd-47ce-a3bc-07c266a4750d None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:38.283362 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c6b0bca2-2bfd-47ce-a3bc-07c266a4750d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:38.296367 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b0bca2-2bfd-47ce-a3bc-07c266a4750d None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:38.302138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114ff31-566f-4017-9a45-7232d6979be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:38.303051 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114ff31-566f-4017-9a45-7232d6979be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:38.303420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114ff31-566f-4017-9a45-7232d6979be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:38.303758 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114ff31-566f-4017-9a45-7232d6979be2 admin admin] PATH_INFO: `/v3/projects/6b680ed7148a494e94972833dc2348e2` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:38.304950 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114ff31-566f-4017-9a45-7232d6979be2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b680ed7148a494e94972833dc2348e2)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:38.306986 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114ff31-566f-4017-9a45-7232d6979be2 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:38.360550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f114ff31-566f-4017-9a45-7232d6979be2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b680ed7148a494e94972833dc2348e2', 'request_id': 'req-f114ff31-566f-4017-9a45-7232d6979be2'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:38.368238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f114ff31-566f-4017-9a45-7232d6979be2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b680ed7148a494e94972833dc2348e2', 'request_id': 'req-f114ff31-566f-4017-9a45-7232d6979be2'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:38.374100 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f114ff31-566f-4017-9a45-7232d6979be2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b680ed7148a494e94972833dc2348e2', 'request_id': 'req-f114ff31-566f-4017-9a45-7232d6979be2'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:38.385419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f114ff31-566f-4017-9a45-7232d6979be2 admin admin] The token cache is being invalidate because project 6b680ed7148a494e94972833dc2348e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:10:38.385703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f114ff31-566f-4017-9a45-7232d6979be2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:38.386343 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f114ff31-566f-4017-9a45-7232d6979be2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:38.387852 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1065/2089] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:10:38 2022] DELETE /identity/v3/projects/6b680ed7148a494e94972833dc2348e2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:38.392324 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75837596-506d-4d46-9736-12ec885afdd0 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:38.401834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75837596-506d-4d46-9736-12ec885afdd0 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:38.458736 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8bb94-b72e-4654-b30f-d8ece2f8bfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:38.459409 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8bb94-b72e-4654-b30f-d8ece2f8bfad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:38.459685 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8bb94-b72e-4654-b30f-d8ece2f8bfad admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:38.459952 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c8bb94-b72e-4654-b30f-d8ece2f8bfad admin admin] PATH_INFO: `/v3/users/a1f288cc47c544c6be3693a589784c6f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:38.461069 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8bb94-b72e-4654-b30f-d8ece2f8bfad admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1f288cc47c544c6be3693a589784c6f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:38.462979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c8bb94-b72e-4654-b30f-d8ece2f8bfad admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:38.511696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-19c8bb94-b72e-4654-b30f-d8ece2f8bfad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1f288cc47c544c6be3693a589784c6f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:38.512347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-19c8bb94-b72e-4654-b30f-d8ece2f8bfad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1f288cc47c544c6be3693a589784c6f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:38.521413 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:38.527020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-19c8bb94-b72e-4654-b30f-d8ece2f8bfad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1f288cc47c544c6be3693a589784c6f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:38.535591 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-19c8bb94-b72e-4654-b30f-d8ece2f8bfad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1f288cc47c544c6be3693a589784c6f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:38.536404 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-19c8bb94-b72e-4654-b30f-d8ece2f8bfad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1f288cc47c544c6be3693a589784c6f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:38.548900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1025/2090] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:10:38 2022] DELETE /identity/v3/users/a1f288cc47c544c6be3693a589784c6f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:38.826971 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114ff31-566f-4017-9a45-7232d6979be2 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:38.828053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f114ff31-566f-4017-9a45-7232d6979be2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:38.845363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114ff31-566f-4017-9a45-7232d6979be2 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:38.911817 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8bb94-b72e-4654-b30f-d8ece2f8bfad admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:38.913878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-19c8bb94-b72e-4654-b30f-d8ece2f8bfad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:38.916725 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5829c6bc-e37f-437d-a760-08a293336aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:38.917432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829c6bc-e37f-437d-a760-08a293336aa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:38.917777 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829c6bc-e37f-437d-a760-08a293336aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:38.918117 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5829c6bc-e37f-437d-a760-08a293336aa7 admin admin] PATH_INFO: `/v3/projects/611bc08405cd4af1ba87230bb1c704c1` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:38.919375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829c6bc-e37f-437d-a760-08a293336aa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=611bc08405cd4af1ba87230bb1c704c1)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:38.921400 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5829c6bc-e37f-437d-a760-08a293336aa7 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:38.928804 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c8bb94-b72e-4654-b30f-d8ece2f8bfad admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:38.934288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2df930-3ec1-42eb-8e50-001e9af78d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:38.934972 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2df930-3ec1-42eb-8e50-001e9af78d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:38.935419 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2df930-3ec1-42eb-8e50-001e9af78d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:38.935688 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2df930-3ec1-42eb-8e50-001e9af78d6f admin admin] PATH_INFO: `/v3/users/cde19b5db1ea459499f8c5e7db4ac102` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:38.936922 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2df930-3ec1-42eb-8e50-001e9af78d6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cde19b5db1ea459499f8c5e7db4ac102)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:38.939451 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2df930-3ec1-42eb-8e50-001e9af78d6f admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:38.995330 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5829c6bc-e37f-437d-a760-08a293336aa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '611bc08405cd4af1ba87230bb1c704c1', 'request_id': 'req-5829c6bc-e37f-437d-a760-08a293336aa7'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:39.004608 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5829c6bc-e37f-437d-a760-08a293336aa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '611bc08405cd4af1ba87230bb1c704c1', 'request_id': 'req-5829c6bc-e37f-437d-a760-08a293336aa7'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:39.005401 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4c2df930-3ec1-42eb-8e50-001e9af78d6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cde19b5db1ea459499f8c5e7db4ac102'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:39.006094 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4c2df930-3ec1-42eb-8e50-001e9af78d6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cde19b5db1ea459499f8c5e7db4ac102'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:39.010973 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5829c6bc-e37f-437d-a760-08a293336aa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '611bc08405cd4af1ba87230bb1c704c1', 'request_id': 'req-5829c6bc-e37f-437d-a760-08a293336aa7'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:39.013625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:39.032837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4c2df930-3ec1-42eb-8e50-001e9af78d6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cde19b5db1ea459499f8c5e7db4ac102'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:39.033139 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5829c6bc-e37f-437d-a760-08a293336aa7 admin admin] The token cache is being invalidate because project 611bc08405cd4af1ba87230bb1c704c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:10:39.033139 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5829c6bc-e37f-437d-a760-08a293336aa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:39.033139 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5829c6bc-e37f-437d-a760-08a293336aa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:39.033424 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4c2df930-3ec1-42eb-8e50-001e9af78d6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cde19b5db1ea459499f8c5e7db4ac102'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:39.033424 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4c2df930-3ec1-42eb-8e50-001e9af78d6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cde19b5db1ea459499f8c5e7db4ac102'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:39.033585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1066/2091] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:10:38 2022] DELETE /identity/v3/projects/611bc08405cd4af1ba87230bb1c704c1 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:39.036545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567dd73-6968-43a1-b49c-ee81cae1e9ed None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:39.036816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567dd73-6968-43a1-b49c-ee81cae1e9ed None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:39.037150 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7567dd73-6968-43a1-b49c-ee81cae1e9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:39.049238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1026/2092] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:10:38 2022] DELETE /identity/v3/users/cde19b5db1ea459499f8c5e7db4ac102 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:39.053714 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858c78b-6567-4844-aed3-a08446600e06 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:39.054064 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858c78b-6567-4844-aed3-a08446600e06 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:39.054561 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858c78b-6567-4844-aed3-a08446600e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:39.059587 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-7567dd73-6968-43a1-b49c-ee81cae1e9ed None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:39.060787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-7567dd73-6968-43a1-b49c-ee81cae1e9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:39.076374 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-3858c78b-6567-4844-aed3-a08446600e06 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:39.077780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-3858c78b-6567-4844-aed3-a08446600e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:39.133759 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1067/2093] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:10:39.142115 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e02bd-0ee2-4c06-923d-2d6f89892afc None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:39.142513 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e02bd-0ee2-4c06-923d-2d6f89892afc None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:39.143478 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e02bd-0ee2-4c06-923d-2d6f89892afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:39.155329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1027/2094] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:39.162498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858c78b-6567-4844-aed3-a08446600e06 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:39.164045 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-3858c78b-6567-4844-aed3-a08446600e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:39.164164 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-4a1e02bd-0ee2-4c06-923d-2d6f89892afc None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:39.165520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-4a1e02bd-0ee2-4c06-923d-2d6f89892afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:39.174514 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1068/2095] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:10:39.178684 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e02bd-0ee2-4c06-923d-2d6f89892afc None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:39.189368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e02bd-0ee2-4c06-923d-2d6f89892afc None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:39.195100 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f40eb-4b72-4b59-a505-d4c2b783105b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:39.195713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f40eb-4b72-4b59-a505-d4c2b783105b admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:39.195989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f40eb-4b72-4b59-a505-d4c2b783105b admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:39.196255 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f40eb-4b72-4b59-a505-d4c2b783105b admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:39.196897 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f40eb-4b72-4b59-a505-d4c2b783105b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:10:39.197192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f40eb-4b72-4b59-a505-d4c2b783105b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:39.199036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f40eb-4b72-4b59-a505-d4c2b783105b admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:39.211481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1069/2096] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:10:39.215918 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f40eb-4b72-4b59-a505-d4c2b783105b admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:39.216690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:39.226267 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f40eb-4b72-4b59-a505-d4c2b783105b admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:39.231524 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60659e51-546b-4f0d-8c17-f09d7f184e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:39.232128 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60659e51-546b-4f0d-8c17-f09d7f184e4e admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:39.232404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60659e51-546b-4f0d-8c17-f09d7f184e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:39.232706 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60659e51-546b-4f0d-8c17-f09d7f184e4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:39.233298 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60659e51-546b-4f0d-8c17-f09d7f184e4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:39.234415 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858c78b-6567-4844-aed3-a08446600e06 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:39.236972 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60659e51-546b-4f0d-8c17-f09d7f184e4e admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:39.239417 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385f0ca-c9d3-4f34-a6d8-79bbc394f1e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:39.239977 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385f0ca-c9d3-4f34-a6d8-79bbc394f1e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:39.240291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385f0ca-c9d3-4f34-a6d8-79bbc394f1e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:39.240529 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385f0ca-c9d3-4f34-a6d8-79bbc394f1e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:39.256788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385f0ca-c9d3-4f34-a6d8-79bbc394f1e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:39.258868 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b385f0ca-c9d3-4f34-a6d8-79bbc394f1e0 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:39.270999 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1028/2097] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:10:39.278635 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1070/2098] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:10:39 2022] POST /identity/v3/projects => generated 367 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:10:39.283327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385f0ca-c9d3-4f34-a6d8-79bbc394f1e0 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:39.293482 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385f0ca-c9d3-4f34-a6d8-79bbc394f1e0 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:39.366432 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42445a88-c489-41bb-aeef-9d907fc8e08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:39.367048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42445a88-c489-41bb-aeef-9d907fc8e08b admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:39.367361 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42445a88-c489-41bb-aeef-9d907fc8e08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:39.367637 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42445a88-c489-41bb-aeef-9d907fc8e08b admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:39.368267 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42445a88-c489-41bb-aeef-9d907fc8e08b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:39.370180 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42445a88-c489-41bb-aeef-9d907fc8e08b admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:39.396268 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1029/2099] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:10:39 2022] POST /identity/v3/users => generated 284 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:10:39.400445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60659e51-546b-4f0d-8c17-f09d7f184e4e admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:39.412651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60659e51-546b-4f0d-8c17-f09d7f184e4e admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:39.420915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964ad7d-096c-4e2e-a704-b0cb4ebcba3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:39.421662 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964ad7d-096c-4e2e-a704-b0cb4ebcba3a admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:39.421790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964ad7d-096c-4e2e-a704-b0cb4ebcba3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:39.422144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4964ad7d-096c-4e2e-a704-b0cb4ebcba3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:39.422892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964ad7d-096c-4e2e-a704-b0cb4ebcba3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:39.424614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4964ad7d-096c-4e2e-a704-b0cb4ebcba3a admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:39.439259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1071/2100] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:39.445223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42445a88-c489-41bb-aeef-9d907fc8e08b admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:39.461652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42445a88-c489-41bb-aeef-9d907fc8e08b admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:39.467043 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03cd7c2-bb2f-463c-80f9-8698a23a26e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:39.467856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03cd7c2-bb2f-463c-80f9-8698a23a26e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:39.468137 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03cd7c2-bb2f-463c-80f9-8698a23a26e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:39.468410 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03cd7c2-bb2f-463c-80f9-8698a23a26e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:39.469053 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03cd7c2-bb2f-463c-80f9-8698a23a26e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:39.471343 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03cd7c2-bb2f-463c-80f9-8698a23a26e6 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:39.481397 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1030/2101] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:39.485518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964ad7d-096c-4e2e-a704-b0cb4ebcba3a admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:39.496239 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4964ad7d-096c-4e2e-a704-b0cb4ebcba3a admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:39.501457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ebf191-6569-4b3c-8c88-5ab7ac1543dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:39.502113 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ebf191-6569-4b3c-8c88-5ab7ac1543dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:39.502390 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ebf191-6569-4b3c-8c88-5ab7ac1543dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:39.502689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ebf191-6569-4b3c-8c88-5ab7ac1543dd admin admin] PATH_INFO: `/v3/projects/cf1b42ddfec74b358dbe43845e3b6b95/users/72f5b7f206374d9e97ae363ed1e12f2e/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:39.518924 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ebf191-6569-4b3c-8c88-5ab7ac1543dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf1b42ddfec74b358dbe43845e3b6b95, user_id=72f5b7f206374d9e97ae363ed1e12f2e, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:39.521688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ebf191-6569-4b3c-8c88-5ab7ac1543dd admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:39.534096 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1072/2102] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:39 2022] PUT /identity/v3/projects/cf1b42ddfec74b358dbe43845e3b6b95/users/72f5b7f206374d9e97ae363ed1e12f2e/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:39.544236 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c44855-2480-4018-ac47-84cc75114a81 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:39.544522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c44855-2480-4018-ac47-84cc75114a81 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:39.544792 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c44855-2480-4018-ac47-84cc75114a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:39.561807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ebf191-6569-4b3c-8c88-5ab7ac1543dd admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:39.564564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-77ebf191-6569-4b3c-8c88-5ab7ac1543dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:39.580898 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-d6c44855-2480-4018-ac47-84cc75114a81 None None] MFA Rules not processed for user `72f5b7f206374d9e97ae363ed1e12f2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:39.582144 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-d6c44855-2480-4018-ac47-84cc75114a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:39.636038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ebf191-6569-4b3c-8c88-5ab7ac1543dd admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:39.640583 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0363e-d458-4008-9f5b-4f6109e75e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:39.641759 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0363e-d458-4008-9f5b-4f6109e75e1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:39.642042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0363e-d458-4008-9f5b-4f6109e75e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:39.642307 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f0363e-d458-4008-9f5b-4f6109e75e1f admin admin] PATH_INFO: `/v3/projects/198bf26f4a924b268ece1bc5885bf838` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:39.643438 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0363e-d458-4008-9f5b-4f6109e75e1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=198bf26f4a924b268ece1bc5885bf838)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:39.645337 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f0363e-d458-4008-9f5b-4f6109e75e1f admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:39.652255 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1031/2103] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:39.656700 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c44855-2480-4018-ac47-84cc75114a81 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:39.666822 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c44855-2480-4018-ac47-84cc75114a81 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:39.669090 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:39.671578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619659b9-cb6b-4e1f-8fa6-85b11af9bb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:39.672272 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619659b9-cb6b-4e1f-8fa6-85b11af9bb23 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:39.672777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619659b9-cb6b-4e1f-8fa6-85b11af9bb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:39.673089 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619659b9-cb6b-4e1f-8fa6-85b11af9bb23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:39.673824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619659b9-cb6b-4e1f-8fa6-85b11af9bb23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:39.676298 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619659b9-cb6b-4e1f-8fa6-85b11af9bb23 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:39.693826 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-86f0363e-d458-4008-9f5b-4f6109e75e1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '198bf26f4a924b268ece1bc5885bf838', 'request_id': 'req-86f0363e-d458-4008-9f5b-4f6109e75e1f'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:39.701005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-86f0363e-d458-4008-9f5b-4f6109e75e1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '198bf26f4a924b268ece1bc5885bf838', 'request_id': 'req-86f0363e-d458-4008-9f5b-4f6109e75e1f'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:39.705631 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1032/2104] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:10:39 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:10:39.707330 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-86f0363e-d458-4008-9f5b-4f6109e75e1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '198bf26f4a924b268ece1bc5885bf838', 'request_id': 'req-86f0363e-d458-4008-9f5b-4f6109e75e1f'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:39.709724 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619659b9-cb6b-4e1f-8fa6-85b11af9bb23 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:39.718462 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-86f0363e-d458-4008-9f5b-4f6109e75e1f admin admin] The token cache is being invalidate because project 198bf26f4a924b268ece1bc5885bf838 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:10:39.718745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-86f0363e-d458-4008-9f5b-4f6109e75e1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:39.719437 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-86f0363e-d458-4008-9f5b-4f6109e75e1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:39.721646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1073/2105] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:10:39 2022] DELETE /identity/v3/projects/198bf26f4a924b268ece1bc5885bf838 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:10:39.725390 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0363e-d458-4008-9f5b-4f6109e75e1f admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:39.726860 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-86f0363e-d458-4008-9f5b-4f6109e75e1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:39.746771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:39.777039 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619659b9-cb6b-4e1f-8fa6-85b11af9bb23 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:39.781735 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a3af1-932e-41d3-93c5-16426fcd2c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:39.782368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a3af1-932e-41d3-93c5-16426fcd2c4f admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:39.782645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a3af1-932e-41d3-93c5-16426fcd2c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:39.782915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a3af1-932e-41d3-93c5-16426fcd2c4f admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:39.783569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a3af1-932e-41d3-93c5-16426fcd2c4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:39.785467 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a3af1-932e-41d3-93c5-16426fcd2c4f admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:39.792695 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f0363e-d458-4008-9f5b-4f6109e75e1f admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:39.797449 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545572e7-4b9b-4b2a-83e1-0b73c5dc161a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:39.798127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545572e7-4b9b-4b2a-83e1-0b73c5dc161a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:39.798398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545572e7-4b9b-4b2a-83e1-0b73c5dc161a admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:39.798667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545572e7-4b9b-4b2a-83e1-0b73c5dc161a admin admin] PATH_INFO: `/v3/users/0c19928346154e27bf2e8e97361b32ac` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:39.799807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545572e7-4b9b-4b2a-83e1-0b73c5dc161a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c19928346154e27bf2e8e97361b32ac)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:39.801748 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545572e7-4b9b-4b2a-83e1-0b73c5dc161a admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:39.806085 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1033/2106] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:10:39 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:10:39.809920 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a3af1-932e-41d3-93c5-16426fcd2c4f admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:39.811587 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b69a3af1-932e-41d3-93c5-16426fcd2c4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:39.824499 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a3af1-932e-41d3-93c5-16426fcd2c4f admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:39.830319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356aa39-68ce-4f11-9590-c6dc835397de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:39.831154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356aa39-68ce-4f11-9590-c6dc835397de admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:39.831447 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356aa39-68ce-4f11-9590-c6dc835397de admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:39.831719 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356aa39-68ce-4f11-9590-c6dc835397de admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:39.832437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356aa39-68ce-4f11-9590-c6dc835397de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:39.834199 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356aa39-68ce-4f11-9590-c6dc835397de admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:39.844640 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1034/2107] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:10:39.848771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356aa39-68ce-4f11-9590-c6dc835397de admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:39.851494 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-545572e7-4b9b-4b2a-83e1-0b73c5dc161a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c19928346154e27bf2e8e97361b32ac'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:39.858479 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-545572e7-4b9b-4b2a-83e1-0b73c5dc161a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c19928346154e27bf2e8e97361b32ac'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:39.859011 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356aa39-68ce-4f11-9590-c6dc835397de admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:39.864385 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea69db-10e8-452b-bbb9-376d2b663ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:39.865294 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea69db-10e8-452b-bbb9-376d2b663ea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:39.865712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea69db-10e8-452b-bbb9-376d2b663ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:39.865991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea69db-10e8-452b-bbb9-376d2b663ea9 admin admin] PATH_INFO: `/v3/projects/c7a989172b4e426f8dc2f4d2e4591ce2/users/827c5d7f7f674af398c812967041f405/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:39.866177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-545572e7-4b9b-4b2a-83e1-0b73c5dc161a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c19928346154e27bf2e8e97361b32ac'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:39.866932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-545572e7-4b9b-4b2a-83e1-0b73c5dc161a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c19928346154e27bf2e8e97361b32ac'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:39.867585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-545572e7-4b9b-4b2a-83e1-0b73c5dc161a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c19928346154e27bf2e8e97361b32ac'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:39.881532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea69db-10e8-452b-bbb9-376d2b663ea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7a989172b4e426f8dc2f4d2e4591ce2, user_id=827c5d7f7f674af398c812967041f405, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:39.884214 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea69db-10e8-452b-bbb9-376d2b663ea9 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:39.885516 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1074/2108] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:10:39 2022] DELETE /identity/v3/users/0c19928346154e27bf2e8e97361b32ac => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:39.898601 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1035/2109] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:39 2022] PUT /identity/v3/projects/c7a989172b4e426f8dc2f4d2e4591ce2/users/827c5d7f7f674af398c812967041f405/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:10:39.902636 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545572e7-4b9b-4b2a-83e1-0b73c5dc161a admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:39.904167 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-545572e7-4b9b-4b2a-83e1-0b73c5dc161a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:39.937184 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:39.975130 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545572e7-4b9b-4b2a-83e1-0b73c5dc161a admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:40.046786 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2adb15c-f802-403d-8f33-2abb8990621f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:40.047571 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2adb15c-f802-403d-8f33-2abb8990621f admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:40.049950 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2adb15c-f802-403d-8f33-2abb8990621f admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:40.050748 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2adb15c-f802-403d-8f33-2abb8990621f admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:40.055486 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2adb15c-f802-403d-8f33-2abb8990621f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:40.057189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2adb15c-f802-403d-8f33-2abb8990621f admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:40.069423 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1075/2110] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:40.073422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea69db-10e8-452b-bbb9-376d2b663ea9 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:40.087776 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea69db-10e8-452b-bbb9-376d2b663ea9 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:40.099629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b128d-9d54-4de1-a298-3fbcd32bb527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:40.100585 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b128d-9d54-4de1-a298-3fbcd32bb527 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:40.100966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b128d-9d54-4de1-a298-3fbcd32bb527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:40.101337 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b128d-9d54-4de1-a298-3fbcd32bb527 admin admin] PATH_INFO: `/v3/projects/c7a989172b4e426f8dc2f4d2e4591ce2/users/827c5d7f7f674af398c812967041f405/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:40.103439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b128d-9d54-4de1-a298-3fbcd32bb527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7a989172b4e426f8dc2f4d2e4591ce2, user_id=827c5d7f7f674af398c812967041f405, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:40.106217 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b128d-9d54-4de1-a298-3fbcd32bb527 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:40.117947 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1036/2111] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:40 2022] PUT /identity/v3/projects/c7a989172b4e426f8dc2f4d2e4591ce2/users/827c5d7f7f674af398c812967041f405/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:40.128942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a87f73-28d0-4e37-86b7-e3c5b3924fb4 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:40.129226 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a87f73-28d0-4e37-86b7-e3c5b3924fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:40.129510 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a87f73-28d0-4e37-86b7-e3c5b3924fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:40.159213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-e5a87f73-28d0-4e37-86b7-e3c5b3924fb4 None None] MFA Rules not processed for user `827c5d7f7f674af398c812967041f405`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:40.160421 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-e5a87f73-28d0-4e37-86b7-e3c5b3924fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:40.200534 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b128d-9d54-4de1-a298-3fbcd32bb527 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:40.202363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-4f0b128d-9d54-4de1-a298-3fbcd32bb527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:40.247916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1076/2112] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:40.260720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a87f73-28d0-4e37-86b7-e3c5b3924fb4 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:40.262470 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-e5a87f73-28d0-4e37-86b7-e3c5b3924fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:40.284075 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b128d-9d54-4de1-a298-3fbcd32bb527 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:40.288762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0340190-d626-4db5-a4e0-9b51e92added admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:40.289392 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0340190-d626-4db5-a4e0-9b51e92added admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:40.289680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0340190-d626-4db5-a4e0-9b51e92added admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:40.289950 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0340190-d626-4db5-a4e0-9b51e92added admin admin] PATH_INFO: `/v3/projects/a3d771ee834f469681d3da1eb2478139` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:40.291204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0340190-d626-4db5-a4e0-9b51e92added admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3d771ee834f469681d3da1eb2478139)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:40.293101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0340190-d626-4db5-a4e0-9b51e92added admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:40.318661 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a87f73-28d0-4e37-86b7-e3c5b3924fb4 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:40.322428 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37677f23-e37f-4359-81d8-c1dea0997c8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:40.323026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37677f23-e37f-4359-81d8-c1dea0997c8c service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:40.323341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37677f23-e37f-4359-81d8-c1dea0997c8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:40.323861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37677f23-e37f-4359-81d8-c1dea0997c8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:40.332726 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37677f23-e37f-4359-81d8-c1dea0997c8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:40.334151 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37677f23-e37f-4359-81d8-c1dea0997c8c service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:40.346081 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:40.347320 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b0340190-d626-4db5-a4e0-9b51e92added admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3d771ee834f469681d3da1eb2478139', 'request_id': 'req-b0340190-d626-4db5-a4e0-9b51e92added'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:40.354094 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b0340190-d626-4db5-a4e0-9b51e92added admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3d771ee834f469681d3da1eb2478139', 'request_id': 'req-b0340190-d626-4db5-a4e0-9b51e92added'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:40.357569 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1077/2113] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:10:40.362347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b0340190-d626-4db5-a4e0-9b51e92added admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3d771ee834f469681d3da1eb2478139', 'request_id': 'req-b0340190-d626-4db5-a4e0-9b51e92added'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:40.374865 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b0340190-d626-4db5-a4e0-9b51e92added admin admin] The token cache is being invalidate because project a3d771ee834f469681d3da1eb2478139 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:10:40.375191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b0340190-d626-4db5-a4e0-9b51e92added admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:40.375931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b0340190-d626-4db5-a4e0-9b51e92added admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:40.378906 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1037/2114] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:10:40 2022] DELETE /identity/v3/projects/a3d771ee834f469681d3da1eb2478139 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:40.383273 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37677f23-e37f-4359-81d8-c1dea0997c8c service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:40.394156 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37677f23-e37f-4359-81d8-c1dea0997c8c service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:40.405763 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0340190-d626-4db5-a4e0-9b51e92added admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:40.407695 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b0340190-d626-4db5-a4e0-9b51e92added admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:40.413034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:40.458095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446fc7a-d9d4-4891-a526-cba2f9779e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:40.458618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446fc7a-d9d4-4891-a526-cba2f9779e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:40.458898 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446fc7a-d9d4-4891-a526-cba2f9779e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:40.459318 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446fc7a-d9d4-4891-a526-cba2f9779e80 admin admin] PATH_INFO: `/v3/users/9574fff7edda41f2b466d6ec519445f9` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:40.460846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446fc7a-d9d4-4891-a526-cba2f9779e80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9574fff7edda41f2b466d6ec519445f9)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:40.465378 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446fc7a-d9d4-4891-a526-cba2f9779e80 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:40.467490 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0340190-d626-4db5-a4e0-9b51e92added admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:40.472383 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3728a04-adfb-4604-92c3-0a88eaeab5f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:40.472998 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3728a04-adfb-4604-92c3-0a88eaeab5f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:40.473274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3728a04-adfb-4604-92c3-0a88eaeab5f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:40.473543 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3728a04-adfb-4604-92c3-0a88eaeab5f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:40.475479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c3728a04-adfb-4604-92c3-0a88eaeab5f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:40.526392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-1446fc7a-d9d4-4891-a526-cba2f9779e80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9574fff7edda41f2b466d6ec519445f9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:40.534935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-1446fc7a-d9d4-4891-a526-cba2f9779e80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9574fff7edda41f2b466d6ec519445f9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:40.543492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-1446fc7a-d9d4-4891-a526-cba2f9779e80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9574fff7edda41f2b466d6ec519445f9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:40.544506 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-1446fc7a-d9d4-4891-a526-cba2f9779e80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9574fff7edda41f2b466d6ec519445f9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:40.545078 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-1446fc7a-d9d4-4891-a526-cba2f9779e80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9574fff7edda41f2b466d6ec519445f9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:40.548359 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3728a04-adfb-4604-92c3-0a88eaeab5f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:40.549798 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3728a04-adfb-4604-92c3-0a88eaeab5f1 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:40.569189 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1038/2115] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:10:40.572671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1078/2116] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:10:40 2022] DELETE /identity/v3/users/9574fff7edda41f2b466d6ec519445f9 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:10:41.101446 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3728a04-adfb-4604-92c3-0a88eaeab5f1 service cinder] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:41.102901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c3728a04-adfb-4604-92c3-0a88eaeab5f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:41.117698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3728a04-adfb-4604-92c3-0a88eaeab5f1 service cinder] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:41.182818 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9b105-2d81-4710-8ce3-a61b0432bb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:41.183484 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9b105-2d81-4710-8ce3-a61b0432bb41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:41.183765 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9b105-2d81-4710-8ce3-a61b0432bb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:41.184041 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9b105-2d81-4710-8ce3-a61b0432bb41 admin admin] PATH_INFO: `/v3/projects/3b1529dc077545999f0e454334a4c8eb` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:41.185178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9b105-2d81-4710-8ce3-a61b0432bb41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b1529dc077545999f0e454334a4c8eb)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:41.187107 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9b105-2d81-4710-8ce3-a61b0432bb41 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:41.234214 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-53b9b105-2d81-4710-8ce3-a61b0432bb41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b1529dc077545999f0e454334a4c8eb', 'request_id': 'req-53b9b105-2d81-4710-8ce3-a61b0432bb41'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:41.241937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-53b9b105-2d81-4710-8ce3-a61b0432bb41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b1529dc077545999f0e454334a4c8eb', 'request_id': 'req-53b9b105-2d81-4710-8ce3-a61b0432bb41'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:41.246051 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:41.248375 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-53b9b105-2d81-4710-8ce3-a61b0432bb41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b1529dc077545999f0e454334a4c8eb', 'request_id': 'req-53b9b105-2d81-4710-8ce3-a61b0432bb41'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:41.263223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-53b9b105-2d81-4710-8ce3-a61b0432bb41 admin admin] The token cache is being invalidate because project 3b1529dc077545999f0e454334a4c8eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:10:41.263546 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-53b9b105-2d81-4710-8ce3-a61b0432bb41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:41.264178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-53b9b105-2d81-4710-8ce3-a61b0432bb41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:41.269667 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1039/2117] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:10:41 2022] DELETE /identity/v3/projects/3b1529dc077545999f0e454334a4c8eb => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:41.271478 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364a736-757d-4d5d-b96a-f4c96ba5a4e5 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:41.271976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364a736-757d-4d5d-b96a-f4c96ba5a4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:41.272160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364a736-757d-4d5d-b96a-f4c96ba5a4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:41.292587 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-2364a736-757d-4d5d-b96a-f4c96ba5a4e5 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:41.293807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-2364a736-757d-4d5d-b96a-f4c96ba5a4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:41.361813 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1079/2118] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:41.369587 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45952262-98bd-4e10-b8f5-cd49c7385fa8 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:41.369877 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45952262-98bd-4e10-b8f5-cd49c7385fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:41.370152 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45952262-98bd-4e10-b8f5-cd49c7385fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:41.389654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-45952262-98bd-4e10-b8f5-cd49c7385fa8 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:41.390883 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-45952262-98bd-4e10-b8f5-cd49c7385fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:41.399645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1040/2119] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:41.403764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364a736-757d-4d5d-b96a-f4c96ba5a4e5 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:41.413944 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364a736-757d-4d5d-b96a-f4c96ba5a4e5 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:41.418478 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6bae3-f04e-46fa-9981-eb375f296135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:41.419089 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6bae3-f04e-46fa-9981-eb375f296135 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:41.419376 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6bae3-f04e-46fa-9981-eb375f296135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:41.419648 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6bae3-f04e-46fa-9981-eb375f296135 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:41.420282 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6bae3-f04e-46fa-9981-eb375f296135 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:10:41.420582 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6bae3-f04e-46fa-9981-eb375f296135 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:41.422170 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b6bae3-f04e-46fa-9981-eb375f296135 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:41.435465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1080/2120] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:10:41.439612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45952262-98bd-4e10-b8f5-cd49c7385fa8 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:41.449105 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45952262-98bd-4e10-b8f5-cd49c7385fa8 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:41.454472 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d1ff7-285c-4144-9206-6e7fb4c44191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:41.455114 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d1ff7-285c-4144-9206-6e7fb4c44191 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:41.455399 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d1ff7-285c-4144-9206-6e7fb4c44191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:41.455671 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d1ff7-285c-4144-9206-6e7fb4c44191 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:41.456310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d1ff7-285c-4144-9206-6e7fb4c44191 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:41.458221 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d1ff7-285c-4144-9206-6e7fb4c44191 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:41.486147 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1041/2121] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:10:41 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:41.490314 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6bae3-f04e-46fa-9981-eb375f296135 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:41.501111 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6bae3-f04e-46fa-9981-eb375f296135 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:41.504702 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:41.558927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d3542-ed08-4642-87dd-e2fdcfcd5029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:41.559480 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d3542-ed08-4642-87dd-e2fdcfcd5029 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:41.559758 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d3542-ed08-4642-87dd-e2fdcfcd5029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:41.560029 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d3542-ed08-4642-87dd-e2fdcfcd5029 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:41.560664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d3542-ed08-4642-87dd-e2fdcfcd5029 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:41.562610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d3542-ed08-4642-87dd-e2fdcfcd5029 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:41.585044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1081/2122] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:10:41 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:41.589280 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d1ff7-285c-4144-9206-6e7fb4c44191 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:41.599947 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d1ff7-285c-4144-9206-6e7fb4c44191 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:41.604744 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373cc3a-5b26-418a-95fb-faaab35cf1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:41.605334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373cc3a-5b26-418a-95fb-faaab35cf1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:41.605615 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373cc3a-5b26-418a-95fb-faaab35cf1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:41.605888 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d373cc3a-5b26-418a-95fb-faaab35cf1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:41.606530 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373cc3a-5b26-418a-95fb-faaab35cf1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:41.608146 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d373cc3a-5b26-418a-95fb-faaab35cf1d8 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:41.617762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1042/2123] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:41.621467 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d3542-ed08-4642-87dd-e2fdcfcd5029 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:41.631267 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d3542-ed08-4642-87dd-e2fdcfcd5029 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:41.635831 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8838cabc-8706-4f38-9969-8cd6083f6473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:41.636415 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838cabc-8706-4f38-9969-8cd6083f6473 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:41.636691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838cabc-8706-4f38-9969-8cd6083f6473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:41.636962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8838cabc-8706-4f38-9969-8cd6083f6473 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:41.637590 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838cabc-8706-4f38-9969-8cd6083f6473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:41.639201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8838cabc-8706-4f38-9969-8cd6083f6473 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:41.650043 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1082/2124] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:41.653959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373cc3a-5b26-418a-95fb-faaab35cf1d8 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:41.663660 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d373cc3a-5b26-418a-95fb-faaab35cf1d8 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:41.668292 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c77c0f-c3dc-4f54-8648-994d6584a992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:41.668937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c77c0f-c3dc-4f54-8648-994d6584a992 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:41.669218 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c77c0f-c3dc-4f54-8648-994d6584a992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:41.669498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c77c0f-c3dc-4f54-8648-994d6584a992 admin admin] PATH_INFO: `/v3/projects/d81e4ab123d54ebdbf4165e250967df6/users/0fd2f1b7788a4095a28f8fb6917c5705/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:41.686436 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c77c0f-c3dc-4f54-8648-994d6584a992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d81e4ab123d54ebdbf4165e250967df6, user_id=0fd2f1b7788a4095a28f8fb6917c5705, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:41.689112 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c77c0f-c3dc-4f54-8648-994d6584a992 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:41.701896 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1043/2125] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:41 2022] PUT /identity/v3/projects/d81e4ab123d54ebdbf4165e250967df6/users/0fd2f1b7788a4095a28f8fb6917c5705/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:41.707481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff3aad-0946-4b24-b470-2bf1dcb52f0c None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:41.707761 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff3aad-0946-4b24-b470-2bf1dcb52f0c None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:41.708034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff3aad-0946-4b24-b470-2bf1dcb52f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:41.728383 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-b7ff3aad-0946-4b24-b470-2bf1dcb52f0c None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:41.729567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-b7ff3aad-0946-4b24-b470-2bf1dcb52f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:41.793520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1083/2126] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:10:41.803562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c77c0f-c3dc-4f54-8648-994d6584a992 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:41.805245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-29c77c0f-c3dc-4f54-8648-994d6584a992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:41.862562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c77c0f-c3dc-4f54-8648-994d6584a992 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:41.866253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d381278-dc3b-4a15-a55f-969f701182f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:41.866854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d381278-dc3b-4a15-a55f-969f701182f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:41.867193 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d381278-dc3b-4a15-a55f-969f701182f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:41.867467 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d381278-dc3b-4a15-a55f-969f701182f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:41.876770 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d381278-dc3b-4a15-a55f-969f701182f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:41.878222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d381278-dc3b-4a15-a55f-969f701182f0 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:41.887550 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1044/2127] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:10:44.396121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff3aad-0946-4b24-b470-2bf1dcb52f0c None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:44.397562 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-b7ff3aad-0946-4b24-b470-2bf1dcb52f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:44.409870 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff3aad-0946-4b24-b470-2bf1dcb52f0c None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:44.414514 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67dce02-9ff7-4d34-bd76-3261fd1b9335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:44.415221 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67dce02-9ff7-4d34-bd76-3261fd1b9335 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:44.415507 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67dce02-9ff7-4d34-bd76-3261fd1b9335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:44.415788 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67dce02-9ff7-4d34-bd76-3261fd1b9335 admin admin] PATH_INFO: `/v3/users/72f5b7f206374d9e97ae363ed1e12f2e` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:44.416936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67dce02-9ff7-4d34-bd76-3261fd1b9335 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72f5b7f206374d9e97ae363ed1e12f2e)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:44.419436 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67dce02-9ff7-4d34-bd76-3261fd1b9335 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:44.471975 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f67dce02-9ff7-4d34-bd76-3261fd1b9335 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72f5b7f206374d9e97ae363ed1e12f2e'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:44.479431 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f67dce02-9ff7-4d34-bd76-3261fd1b9335 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72f5b7f206374d9e97ae363ed1e12f2e'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:44.488376 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f67dce02-9ff7-4d34-bd76-3261fd1b9335 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72f5b7f206374d9e97ae363ed1e12f2e'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:44.489227 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f67dce02-9ff7-4d34-bd76-3261fd1b9335 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72f5b7f206374d9e97ae363ed1e12f2e'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:44.489952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f67dce02-9ff7-4d34-bd76-3261fd1b9335 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72f5b7f206374d9e97ae363ed1e12f2e'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:44.506580 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:44.512822 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1084/2128] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:10:44 2022] DELETE /identity/v3/users/72f5b7f206374d9e97ae363ed1e12f2e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:44.517859 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674b79c-37c4-448b-a396-b624269799c4 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:44.518471 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674b79c-37c4-448b-a396-b624269799c4 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:44.518471 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674b79c-37c4-448b-a396-b624269799c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:44.538589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-2674b79c-37c4-448b-a396-b624269799c4 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:44.539912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-2674b79c-37c4-448b-a396-b624269799c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:44.604625 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1045/2129] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:44.614310 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67dce02-9ff7-4d34-bd76-3261fd1b9335 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:44.615798 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f67dce02-9ff7-4d34-bd76-3261fd1b9335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:44.710595 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67dce02-9ff7-4d34-bd76-3261fd1b9335 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:44.718420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff19f99-d75b-4e76-8ea3-474d83703fed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:44.719329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff19f99-d75b-4e76-8ea3-474d83703fed service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:44.719731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff19f99-d75b-4e76-8ea3-474d83703fed service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:44.720107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff19f99-d75b-4e76-8ea3-474d83703fed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:44.736180 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff19f99-d75b-4e76-8ea3-474d83703fed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:44.737623 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff19f99-d75b-4e76-8ea3-474d83703fed service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:44.748650 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1085/2130] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:10:45.223848 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674b79c-37c4-448b-a396-b624269799c4 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:45.226298 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-2674b79c-37c4-448b-a396-b624269799c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:45.249214 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674b79c-37c4-448b-a396-b624269799c4 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:45.255461 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f10fa-4bc9-48c5-8150-d46bdfdf4cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:45.256818 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f10fa-4bc9-48c5-8150-d46bdfdf4cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:45.257205 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f10fa-4bc9-48c5-8150-d46bdfdf4cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:45.257609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f10fa-4bc9-48c5-8150-d46bdfdf4cdd admin admin] PATH_INFO: `/v3/projects/cf1b42ddfec74b358dbe43845e3b6b95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:45.259175 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f10fa-4bc9-48c5-8150-d46bdfdf4cdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf1b42ddfec74b358dbe43845e3b6b95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:45.261751 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f10fa-4bc9-48c5-8150-d46bdfdf4cdd admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:45.297920 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:45.356014 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-197f10fa-4bc9-48c5-8150-d46bdfdf4cdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf1b42ddfec74b358dbe43845e3b6b95', 'request_id': 'req-197f10fa-4bc9-48c5-8150-d46bdfdf4cdd'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:45.366983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-197f10fa-4bc9-48c5-8150-d46bdfdf4cdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf1b42ddfec74b358dbe43845e3b6b95', 'request_id': 'req-197f10fa-4bc9-48c5-8150-d46bdfdf4cdd'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:45.375002 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-197f10fa-4bc9-48c5-8150-d46bdfdf4cdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf1b42ddfec74b358dbe43845e3b6b95', 'request_id': 'req-197f10fa-4bc9-48c5-8150-d46bdfdf4cdd'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:45.400974 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-197f10fa-4bc9-48c5-8150-d46bdfdf4cdd admin admin] The token cache is being invalidate because project cf1b42ddfec74b358dbe43845e3b6b95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:10:45.401189 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-197f10fa-4bc9-48c5-8150-d46bdfdf4cdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:45.402088 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-197f10fa-4bc9-48c5-8150-d46bdfdf4cdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:45.406775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1046/2131] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:10:45 2022] DELETE /identity/v3/projects/cf1b42ddfec74b358dbe43845e3b6b95 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:45.408357 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff19f99-d75b-4e76-8ea3-474d83703fed service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:45.418411 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff19f99-d75b-4e76-8ea3-474d83703fed service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:45.495400 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee046f32-dcb9-4417-bc16-f9ec03989e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:45.496389 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee046f32-dcb9-4417-bc16-f9ec03989e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:45.497961 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee046f32-dcb9-4417-bc16-f9ec03989e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:45.498296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee046f32-dcb9-4417-bc16-f9ec03989e0b admin admin] PATH_INFO: `/v3/users/827c5d7f7f674af398c812967041f405` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:45.500080 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee046f32-dcb9-4417-bc16-f9ec03989e0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=827c5d7f7f674af398c812967041f405)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:45.502910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee046f32-dcb9-4417-bc16-f9ec03989e0b admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:45.572519 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ee046f32-dcb9-4417-bc16-f9ec03989e0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '827c5d7f7f674af398c812967041f405'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:45.581586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ee046f32-dcb9-4417-bc16-f9ec03989e0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '827c5d7f7f674af398c812967041f405'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:45.590861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ee046f32-dcb9-4417-bc16-f9ec03989e0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '827c5d7f7f674af398c812967041f405'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:45.592064 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ee046f32-dcb9-4417-bc16-f9ec03989e0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '827c5d7f7f674af398c812967041f405'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:45.592985 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ee046f32-dcb9-4417-bc16-f9ec03989e0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '827c5d7f7f674af398c812967041f405'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:45.618869 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1086/2132] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:10:45 2022] DELETE /identity/v3/users/827c5d7f7f674af398c812967041f405 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:46.075523 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f10fa-4bc9-48c5-8150-d46bdfdf4cdd admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:46.077123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-197f10fa-4bc9-48c5-8150-d46bdfdf4cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:46.100352 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f10fa-4bc9-48c5-8150-d46bdfdf4cdd admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:46.179001 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b352b-f34e-42a1-a367-5b6d743eeb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:46.180014 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b352b-f34e-42a1-a367-5b6d743eeb82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:46.180292 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b352b-f34e-42a1-a367-5b6d743eeb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:46.181014 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b352b-f34e-42a1-a367-5b6d743eeb82 admin admin] PATH_INFO: `/v3/projects/c7a989172b4e426f8dc2f4d2e4591ce2` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:46.182647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b352b-f34e-42a1-a367-5b6d743eeb82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7a989172b4e426f8dc2f4d2e4591ce2)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:46.184782 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535b352b-f34e-42a1-a367-5b6d743eeb82 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:46.253201 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-535b352b-f34e-42a1-a367-5b6d743eeb82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7a989172b4e426f8dc2f4d2e4591ce2', 'request_id': 'req-535b352b-f34e-42a1-a367-5b6d743eeb82'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:46.262249 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-535b352b-f34e-42a1-a367-5b6d743eeb82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7a989172b4e426f8dc2f4d2e4591ce2', 'request_id': 'req-535b352b-f34e-42a1-a367-5b6d743eeb82'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:46.267859 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-535b352b-f34e-42a1-a367-5b6d743eeb82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7a989172b4e426f8dc2f4d2e4591ce2', 'request_id': 'req-535b352b-f34e-42a1-a367-5b6d743eeb82'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:46.285356 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-535b352b-f34e-42a1-a367-5b6d743eeb82 admin admin] The token cache is being invalidate because project c7a989172b4e426f8dc2f4d2e4591ce2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:10:46.286505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-535b352b-f34e-42a1-a367-5b6d743eeb82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:46.287307 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-535b352b-f34e-42a1-a367-5b6d743eeb82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:10:46.295857 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1047/2133] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:10:46 2022] DELETE /identity/v3/projects/c7a989172b4e426f8dc2f4d2e4591ce2 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:46.296210 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3621f1f-aa06-4c60-aaaf-e3a6804ca15a None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:46.296500 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3621f1f-aa06-4c60-aaaf-e3a6804ca15a None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:46.296796 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3621f1f-aa06-4c60-aaaf-e3a6804ca15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:46.331383 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-e3621f1f-aa06-4c60-aaaf-e3a6804ca15a None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:46.332317 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-e3621f1f-aa06-4c60-aaaf-e3a6804ca15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:46.348404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:46.403857 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1087/2134] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:10:46.412215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f8177-2ed6-4cc3-b1ba-5fbe711bb5f0 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:46.412564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f8177-2ed6-4cc3-b1ba-5fbe711bb5f0 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:46.413036 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f8177-2ed6-4cc3-b1ba-5fbe711bb5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:46.439390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-720f8177-2ed6-4cc3-b1ba-5fbe711bb5f0 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:46.441940 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-720f8177-2ed6-4cc3-b1ba-5fbe711bb5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:46.452609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1048/2135] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:46.454494 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3621f1f-aa06-4c60-aaaf-e3a6804ca15a None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:46.466783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3621f1f-aa06-4c60-aaaf-e3a6804ca15a None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:46.473988 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97489d-1200-4bf3-a8dd-304bc574b304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:46.475055 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97489d-1200-4bf3-a8dd-304bc574b304 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:46.475762 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97489d-1200-4bf3-a8dd-304bc574b304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:46.475890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97489d-1200-4bf3-a8dd-304bc574b304 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:46.476450 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97489d-1200-4bf3-a8dd-304bc574b304 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:10:46.476761 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97489d-1200-4bf3-a8dd-304bc574b304 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:46.479031 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97489d-1200-4bf3-a8dd-304bc574b304 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:46.498487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1088/2136] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:10:46.502659 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f8177-2ed6-4cc3-b1ba-5fbe711bb5f0 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:46.512941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f8177-2ed6-4cc3-b1ba-5fbe711bb5f0 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:46.522924 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db517f-0f97-48a1-bf05-ec768f427954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:46.523946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db517f-0f97-48a1-bf05-ec768f427954 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:46.524361 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db517f-0f97-48a1-bf05-ec768f427954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:46.524651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13db517f-0f97-48a1-bf05-ec768f427954 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:46.525357 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db517f-0f97-48a1-bf05-ec768f427954 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:46.527523 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13db517f-0f97-48a1-bf05-ec768f427954 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:46.601253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1049/2137] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:10:46 2022] POST /identity/v3/projects => generated 365 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:46.603476 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97489d-1200-4bf3-a8dd-304bc574b304 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:46.623483 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97489d-1200-4bf3-a8dd-304bc574b304 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:46.787195 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c054a-1361-4f6c-b494-1e4e297170cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:46.787956 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c054a-1361-4f6c-b494-1e4e297170cb admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:46.788334 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c054a-1361-4f6c-b494-1e4e297170cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:46.788817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c054a-1361-4f6c-b494-1e4e297170cb admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:46.789810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c054a-1361-4f6c-b494-1e4e297170cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:46.792693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c054a-1361-4f6c-b494-1e4e297170cb admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:46.823699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1089/2138] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:10:46 2022] POST /identity/v3/users => generated 283 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:46.829020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db517f-0f97-48a1-bf05-ec768f427954 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:46.846034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13db517f-0f97-48a1-bf05-ec768f427954 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:46.855547 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9854a1b-6566-4e56-80c4-60ae4aebbe43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:46.856171 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9854a1b-6566-4e56-80c4-60ae4aebbe43 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:46.856523 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9854a1b-6566-4e56-80c4-60ae4aebbe43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:46.856887 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9854a1b-6566-4e56-80c4-60ae4aebbe43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:46.857807 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9854a1b-6566-4e56-80c4-60ae4aebbe43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:46.860123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9854a1b-6566-4e56-80c4-60ae4aebbe43 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:46.867043 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:46.875869 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1050/2139] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:46.877288 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c054a-1361-4f6c-b494-1e4e297170cb admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:46.888562 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c054a-1361-4f6c-b494-1e4e297170cb admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:46.896252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16719f-7336-4570-a2fb-0e9f526a5d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:46.897265 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16719f-7336-4570-a2fb-0e9f526a5d73 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:46.897887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16719f-7336-4570-a2fb-0e9f526a5d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:46.898448 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16719f-7336-4570-a2fb-0e9f526a5d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:46.901875 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16719f-7336-4570-a2fb-0e9f526a5d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:46.902492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16719f-7336-4570-a2fb-0e9f526a5d73 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:46.925703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1090/2140] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:46 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:46.932723 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9854a1b-6566-4e56-80c4-60ae4aebbe43 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:46.945719 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9854a1b-6566-4e56-80c4-60ae4aebbe43 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:46.950755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9c9fe-30f2-4c77-bd1c-853c98a53f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:46.951595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9c9fe-30f2-4c77-bd1c-853c98a53f3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:46.951970 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9c9fe-30f2-4c77-bd1c-853c98a53f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:46.952337 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9c9fe-30f2-4c77-bd1c-853c98a53f3b admin admin] PATH_INFO: `/v3/projects/cab81ac3e4864a939690848e4de734cf/users/5d73ff3d6d27444d88dd6bacd531ce47/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:46.970107 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b9c9fe-30f2-4c77-bd1c-853c98a53f3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cab81ac3e4864a939690848e4de734cf, user_id=5d73ff3d6d27444d88dd6bacd531ce47, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:46.973177 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b9c9fe-30f2-4c77-bd1c-853c98a53f3b admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:46.989572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1051/2141] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:46 2022] PUT /identity/v3/projects/cab81ac3e4864a939690848e4de734cf/users/5d73ff3d6d27444d88dd6bacd531ce47/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:47.000223 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a171a73-6ed4-48eb-90d7-0190adcbb50f None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:47.001278 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a171a73-6ed4-48eb-90d7-0190adcbb50f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:47.001576 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a171a73-6ed4-48eb-90d7-0190adcbb50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:47.035729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-8a171a73-6ed4-48eb-90d7-0190adcbb50f None None] MFA Rules not processed for user `5d73ff3d6d27444d88dd6bacd531ce47`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:47.037285 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8a171a73-6ed4-48eb-90d7-0190adcbb50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:47.123147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1091/2142] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:47.128204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9c9fe-30f2-4c77-bd1c-853c98a53f3b admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:47.204326 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9c9fe-30f2-4c77-bd1c-853c98a53f3b admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:47.209352 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b7879-3d70-4fa6-aa2c-1d94025788c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:47.209997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b7879-3d70-4fa6-aa2c-1d94025788c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:47.210430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b7879-3d70-4fa6-aa2c-1d94025788c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:47.210711 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b7879-3d70-4fa6-aa2c-1d94025788c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:47.211659 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07b7879-3d70-4fa6-aa2c-1d94025788c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:47.213608 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07b7879-3d70-4fa6-aa2c-1d94025788c4 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:47.247805 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a171a73-6ed4-48eb-90d7-0190adcbb50f None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:47.254335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1052/2143] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:10:47 2022] POST /identity/v3/projects => generated 367 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:10:47.263580 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a171a73-6ed4-48eb-90d7-0190adcbb50f None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:47.283697 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:47.404238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ff156-efbf-43ba-9e81-a3540347e442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:47.405850 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ff156-efbf-43ba-9e81-a3540347e442 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:47.406229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ff156-efbf-43ba-9e81-a3540347e442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:47.406638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7ff156-efbf-43ba-9e81-a3540347e442 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:47.407449 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ff156-efbf-43ba-9e81-a3540347e442 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:47.409696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7ff156-efbf-43ba-9e81-a3540347e442 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:47.449320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1092/2144] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:10:47 2022] POST /identity/v3/users => generated 290 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:10:47.450547 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b7879-3d70-4fa6-aa2c-1d94025788c4 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:47.464285 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b7879-3d70-4fa6-aa2c-1d94025788c4 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:47.482912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980da19-30fa-45bf-a5ae-e8ddd32e2073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:47.483610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980da19-30fa-45bf-a5ae-e8ddd32e2073 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:47.486533 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980da19-30fa-45bf-a5ae-e8ddd32e2073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:47.487538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980da19-30fa-45bf-a5ae-e8ddd32e2073 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:47.488288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980da19-30fa-45bf-a5ae-e8ddd32e2073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:47.490265 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980da19-30fa-45bf-a5ae-e8ddd32e2073 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:47.511953 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1053/2145] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:47.516933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ff156-efbf-43ba-9e81-a3540347e442 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:47.527808 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7ff156-efbf-43ba-9e81-a3540347e442 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:47.538319 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010ec115-b853-4bab-a7b9-8ab96e4b664a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:47.539129 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ec115-b853-4bab-a7b9-8ab96e4b664a admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:47.539381 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ec115-b853-4bab-a7b9-8ab96e4b664a admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:47.539655 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010ec115-b853-4bab-a7b9-8ab96e4b664a admin admin] PATH_INFO: `/v3/projects/18c6eff7f04b461d84d7d3ebc445a937/users/b6b6c4910d0747b28cbcccb72c7a6d7e/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:47.558793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ec115-b853-4bab-a7b9-8ab96e4b664a admin admin] RBAC: Authorizing `identity:create_grant(project_id=18c6eff7f04b461d84d7d3ebc445a937, user_id=b6b6c4910d0747b28cbcccb72c7a6d7e, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:47.561940 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010ec115-b853-4bab-a7b9-8ab96e4b664a admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:47.604724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1093/2146] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:47 2022] PUT /identity/v3/projects/18c6eff7f04b461d84d7d3ebc445a937/users/b6b6c4910d0747b28cbcccb72c7a6d7e/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:10:47.609645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980da19-30fa-45bf-a5ae-e8ddd32e2073 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:47.625046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980da19-30fa-45bf-a5ae-e8ddd32e2073 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:47.636255 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0921f8-9988-4fe7-9a1a-06e2bc6e5504 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:47.637087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0921f8-9988-4fe7-9a1a-06e2bc6e5504 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:47.637540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0921f8-9988-4fe7-9a1a-06e2bc6e5504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:47.680671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-ca0921f8-9988-4fe7-9a1a-06e2bc6e5504 None None] MFA Rules not processed for user `0fd2f1b7788a4095a28f8fb6917c5705`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:47.682070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-ca0921f8-9988-4fe7-9a1a-06e2bc6e5504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:47.761325 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8422b8b-e340-4523-af6e-68d908346a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:47.761961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8422b8b-e340-4523-af6e-68d908346a6c admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:47.762243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8422b8b-e340-4523-af6e-68d908346a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:47.762518 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8422b8b-e340-4523-af6e-68d908346a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:47.763337 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8422b8b-e340-4523-af6e-68d908346a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:47.765154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8422b8b-e340-4523-af6e-68d908346a6c admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:47.794522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1054/2147] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:47.796974 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8422b8b-e340-4523-af6e-68d908346a6c admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:47.801705 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1094/2148] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:47.809654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8422b8b-e340-4523-af6e-68d908346a6c admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:47.811540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0921f8-9988-4fe7-9a1a-06e2bc6e5504 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:47.814125 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-ca0921f8-9988-4fe7-9a1a-06e2bc6e5504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:47.820366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7221012-593a-467f-a07a-acd92a794d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:47.821179 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7221012-593a-467f-a07a-acd92a794d8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:47.821558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7221012-593a-467f-a07a-acd92a794d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:47.824777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7221012-593a-467f-a07a-acd92a794d8f admin admin] PATH_INFO: `/v3/projects/18c6eff7f04b461d84d7d3ebc445a937/users/b6b6c4910d0747b28cbcccb72c7a6d7e/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:47.841455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7221012-593a-467f-a07a-acd92a794d8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=18c6eff7f04b461d84d7d3ebc445a937, user_id=b6b6c4910d0747b28cbcccb72c7a6d7e, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:47.841455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7221012-593a-467f-a07a-acd92a794d8f admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:47.847983 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0921f8-9988-4fe7-9a1a-06e2bc6e5504 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:47.855035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86648d5-15fa-4afc-9114-7628e27028d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:47.856567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86648d5-15fa-4afc-9114-7628e27028d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:47.857004 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86648d5-15fa-4afc-9114-7628e27028d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:47.857453 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86648d5-15fa-4afc-9114-7628e27028d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:47.858632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86648d5-15fa-4afc-9114-7628e27028d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:47.862734 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86648d5-15fa-4afc-9114-7628e27028d0 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:47.864180 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1055/2149] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:47 2022] PUT /identity/v3/projects/18c6eff7f04b461d84d7d3ebc445a937/users/b6b6c4910d0747b28cbcccb72c7a6d7e/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:47.878569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a67903-febc-4970-bff4-00c97a604845 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:47.878706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a67903-febc-4970-bff4-00c97a604845 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:47.878908 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a67903-febc-4970-bff4-00c97a604845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:47.906373 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1095/2150] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:10:47 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:47.909016 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86648d5-15fa-4afc-9114-7628e27028d0 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:47.971616 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-78a67903-febc-4970-bff4-00c97a604845 None None] MFA Rules not processed for user `b6b6c4910d0747b28cbcccb72c7a6d7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:47.973868 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-78a67903-febc-4970-bff4-00c97a604845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:48.004468 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:48.024353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:48.035530 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86648d5-15fa-4afc-9114-7628e27028d0 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:48.041114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1dc97c-ba33-449d-b5cd-ecaadcfe8a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:48.041835 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dc97c-ba33-449d-b5cd-ecaadcfe8a0a admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:48.042120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dc97c-ba33-449d-b5cd-ecaadcfe8a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:48.042397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dc97c-ba33-449d-b5cd-ecaadcfe8a0a admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:48.043322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1dc97c-ba33-449d-b5cd-ecaadcfe8a0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:48.045689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1dc97c-ba33-449d-b5cd-ecaadcfe8a0a admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:48.103839 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1096/2151] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:10:47 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:10:48.111731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1dc97c-ba33-449d-b5cd-ecaadcfe8a0a admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:48.132129 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1dc97c-ba33-449d-b5cd-ecaadcfe8a0a admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:48.138527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1056/2152] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:48.140068 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e159b-4a6d-4256-8ed2-7f4bf40469be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:48.140891 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e159b-4a6d-4256-8ed2-7f4bf40469be admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:48.141892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e159b-4a6d-4256-8ed2-7f4bf40469be admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:48.142317 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e159b-4a6d-4256-8ed2-7f4bf40469be admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:48.144392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e159b-4a6d-4256-8ed2-7f4bf40469be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:48.147224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e159b-4a6d-4256-8ed2-7f4bf40469be admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:48.163116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1097/2153] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:48 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:10:48.167540 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a67903-febc-4970-bff4-00c97a604845 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:48.184119 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e159b-4a6d-4256-8ed2-7f4bf40469be admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:48.187458 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a67903-febc-4970-bff4-00c97a604845 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:48.188836 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-e31e159b-4a6d-4256-8ed2-7f4bf40469be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:48.197907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79523633-23aa-441c-873e-5502fa2e58e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:48.198592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79523633-23aa-441c-873e-5502fa2e58e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:48.199172 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79523633-23aa-441c-873e-5502fa2e58e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:48.199442 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79523633-23aa-441c-873e-5502fa2e58e6 admin admin] PATH_INFO: `/v3/projects/d0ce498c6a6e428fa228a314e5c351c5/users/54f2eee57d624e9b8448abc4f4ab685d/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:48.222637 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79523633-23aa-441c-873e-5502fa2e58e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ce498c6a6e428fa228a314e5c351c5, user_id=54f2eee57d624e9b8448abc4f4ab685d, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:48.225597 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79523633-23aa-441c-873e-5502fa2e58e6 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:48.244531 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1057/2154] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:48 2022] PUT /identity/v3/projects/d0ce498c6a6e428fa228a314e5c351c5/users/54f2eee57d624e9b8448abc4f4ab685d/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:48.252785 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79523633-23aa-441c-873e-5502fa2e58e6 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:48.285211 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e159b-4a6d-4256-8ed2-7f4bf40469be admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:48.341737 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79523633-23aa-441c-873e-5502fa2e58e6 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:48.347174 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15851129-cdc1-4469-8967-634ba48795e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:48.347789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15851129-cdc1-4469-8967-634ba48795e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:48.348075 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15851129-cdc1-4469-8967-634ba48795e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:48.348351 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15851129-cdc1-4469-8967-634ba48795e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:48.349000 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15851129-cdc1-4469-8967-634ba48795e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:48.350616 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15851129-cdc1-4469-8967-634ba48795e8 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:48.351709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570867ac-3f63-49bc-852d-15ed37cb85b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:48.352618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570867ac-3f63-49bc-852d-15ed37cb85b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:48.353530 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570867ac-3f63-49bc-852d-15ed37cb85b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:48.354025 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570867ac-3f63-49bc-852d-15ed37cb85b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:48.361486 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1058/2155] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:10:48 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:10:48.365736 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15851129-cdc1-4469-8967-634ba48795e8 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:48.375758 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15851129-cdc1-4469-8967-634ba48795e8 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:48.380378 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de141722-d64c-46e8-ab43-8165722f6405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:48.381165 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de141722-d64c-46e8-ab43-8165722f6405 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:48.381451 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de141722-d64c-46e8-ab43-8165722f6405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:48.381727 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de141722-d64c-46e8-ab43-8165722f6405 admin admin] PATH_INFO: `/v3/projects/d0ce498c6a6e428fa228a314e5c351c5/users/54f2eee57d624e9b8448abc4f4ab685d/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:48.383723 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de141722-d64c-46e8-ab43-8165722f6405 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ce498c6a6e428fa228a314e5c351c5, user_id=54f2eee57d624e9b8448abc4f4ab685d, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:48.386358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de141722-d64c-46e8-ab43-8165722f6405 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:48.397614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1059/2156] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:10:48 2022] PUT /identity/v3/projects/d0ce498c6a6e428fa228a314e5c351c5/users/54f2eee57d624e9b8448abc4f4ab685d/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:10:48.425508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570867ac-3f63-49bc-852d-15ed37cb85b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:48.426956 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570867ac-3f63-49bc-852d-15ed37cb85b6 service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:48.436655 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1098/2157] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:10:48.625189 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de141722-d64c-46e8-ab43-8165722f6405 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:48.707389 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de141722-d64c-46e8-ab43-8165722f6405 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:48.711832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc4f89-b67a-46d6-ab8c-8c7e74f3b8f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:48.712727 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc4f89-b67a-46d6-ab8c-8c7e74f3b8f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:48.713074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc4f89-b67a-46d6-ab8c-8c7e74f3b8f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:48.713460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc4f89-b67a-46d6-ab8c-8c7e74f3b8f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:48.758812 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:48.801397 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc4f89-b67a-46d6-ab8c-8c7e74f3b8f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:48.803110 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc4f89-b67a-46d6-ab8c-8c7e74f3b8f4 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:48.813127 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1060/2158] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:10:53.443729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd60c1-0154-4063-b579-56b411b6432e None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:53.444425 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd60c1-0154-4063-b579-56b411b6432e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:53.444425 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd60c1-0154-4063-b579-56b411b6432e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:53.487448 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-b7cd60c1-0154-4063-b579-56b411b6432e None None] MFA Rules not processed for user `54f2eee57d624e9b8448abc4f4ab685d`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:53.488798 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-b7cd60c1-0154-4063-b579-56b411b6432e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:53.510144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:10:53.588195 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1099/2159] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:53.603466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc4f89-b67a-46d6-ab8c-8c7e74f3b8f4 service cinder] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:10:53.605540 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-aabc4f89-b67a-46d6-ab8c-8c7e74f3b8f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:53.718745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc4f89-b67a-46d6-ab8c-8c7e74f3b8f4 service cinder] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:10:53.725055 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5093f6e-c08a-479f-8278-756198ba34bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:10:53.725675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5093f6e-c08a-479f-8278-756198ba34bb service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:53.725953 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5093f6e-c08a-479f-8278-756198ba34bb service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:53.726225 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5093f6e-c08a-479f-8278-756198ba34bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:53.786104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5093f6e-c08a-479f-8278-756198ba34bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:10:53.787748 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5093f6e-c08a-479f-8278-756198ba34bb service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:10:53.797985 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1061/2160] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:10:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:10:55.139295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59330af3-793a-488b-b0f2-4f8337d3bf99 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:55.139295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59330af3-793a-488b-b0f2-4f8337d3bf99 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:55.139295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59330af3-793a-488b-b0f2-4f8337d3bf99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:55.163118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-59330af3-793a-488b-b0f2-4f8337d3bf99 None None] MFA Rules not processed for user `0fd2f1b7788a4095a28f8fb6917c5705`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:55.164301 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-59330af3-793a-488b-b0f2-4f8337d3bf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:10:55.171881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1100/2161] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:10:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:10:59.946464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc0043-c2c3-4cdd-b6fa-2f0fb31fc159 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:10:59.946975 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc0043-c2c3-4cdd-b6fa-2f0fb31fc159 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:10:59.947185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cc0043-c2c3-4cdd-b6fa-2f0fb31fc159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:10:59.970853 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-30cc0043-c2c3-4cdd-b6fa-2f0fb31fc159 None None] MFA Rules not processed for user `76ebecce283f491e82a961d7f739878f`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:10:59.972361 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-30cc0043-c2c3-4cdd-b6fa-2f0fb31fc159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:00.032116 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1062/2162] 10.222.0.36 () {58 vars in 1009 bytes} [Fri Sep 2 09:10:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:11:00.079978 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59330af3-793a-488b-b0f2-4f8337d3bf99 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:00.083585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-59330af3-793a-488b-b0f2-4f8337d3bf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:00.176160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59330af3-793a-488b-b0f2-4f8337d3bf99 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:00.181866 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f569d7-5312-4cc5-a3ba-fdfe20a9c2c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92318948dcf1407b9fc8aa5c199787ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:00.182477 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f569d7-5312-4cc5-a3ba-fdfe20a9c2c1 service swift] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:00.182757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f569d7-5312-4cc5-a3ba-fdfe20a9c2c1 service swift] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:00.183034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f569d7-5312-4cc5-a3ba-fdfe20a9c2c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:00.193270 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f569d7-5312-4cc5-a3ba-fdfe20a9c2c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:00.194959 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f569d7-5312-4cc5-a3ba-fdfe20a9c2c1 service swift] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:00.205767 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1101/2163] 10.222.0.36 () {60 vars in 1384 bytes} [Fri Sep 2 09:11:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:11:01.273026 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc0043-c2c3-4cdd-b6fa-2f0fb31fc159 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:01.289897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:01.352019 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cc0043-c2c3-4cdd-b6fa-2f0fb31fc159 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:01.357026 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ef744-fb83-40fb-ba41-a41dd73bfdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:01.357821 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ef744-fb83-40fb-ba41-a41dd73bfdc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:01.357942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ef744-fb83-40fb-ba41-a41dd73bfdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:01.358199 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ef744-fb83-40fb-ba41-a41dd73bfdc4 admin admin] PATH_INFO: `/v3/users/5d73ff3d6d27444d88dd6bacd531ce47` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:01.359512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ef744-fb83-40fb-ba41-a41dd73bfdc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d73ff3d6d27444d88dd6bacd531ce47)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:01.362423 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ef744-fb83-40fb-ba41-a41dd73bfdc4 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:01.456651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a84ef744-fb83-40fb-ba41-a41dd73bfdc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d73ff3d6d27444d88dd6bacd531ce47'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:01.458166 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a84ef744-fb83-40fb-ba41-a41dd73bfdc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d73ff3d6d27444d88dd6bacd531ce47'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:01.497459 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a84ef744-fb83-40fb-ba41-a41dd73bfdc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d73ff3d6d27444d88dd6bacd531ce47'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:01.521369 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a84ef744-fb83-40fb-ba41-a41dd73bfdc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d73ff3d6d27444d88dd6bacd531ce47'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:01.524081 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a84ef744-fb83-40fb-ba41-a41dd73bfdc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d73ff3d6d27444d88dd6bacd531ce47'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:01.552948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1063/2164] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:11:01 2022] DELETE /identity/v3/users/5d73ff3d6d27444d88dd6bacd531ce47 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:11:01.557362 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f6020-fa42-4ebf-b12f-7a04f6a02d9f None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:01.558327 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f6020-fa42-4ebf-b12f-7a04f6a02d9f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:01.558823 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9f6020-fa42-4ebf-b12f-7a04f6a02d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:01.622654 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-bb9f6020-fa42-4ebf-b12f-7a04f6a02d9f None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:01.623909 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-bb9f6020-fa42-4ebf-b12f-7a04f6a02d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:01.751161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1102/2165] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:11:01.758367 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ef744-fb83-40fb-ba41-a41dd73bfdc4 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:01.760670 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-a84ef744-fb83-40fb-ba41-a41dd73bfdc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:01.884459 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ef744-fb83-40fb-ba41-a41dd73bfdc4 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:01.890592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1a5db-335d-4683-802a-cf87070b5ad1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:01.891606 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1a5db-335d-4683-802a-cf87070b5ad1 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:01.892092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1a5db-335d-4683-802a-cf87070b5ad1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:01.892603 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1a5db-335d-4683-802a-cf87070b5ad1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:01.908200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c1a5db-335d-4683-802a-cf87070b5ad1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:01.910011 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c1a5db-335d-4683-802a-cf87070b5ad1 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:01.926800 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1064/2166] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:11:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:11:02.444172 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f6020-fa42-4ebf-b12f-7a04f6a02d9f None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:02.446188 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-bb9f6020-fa42-4ebf-b12f-7a04f6a02d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:02.460690 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9f6020-fa42-4ebf-b12f-7a04f6a02d9f None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:02.465638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec2456-52d3-4842-91d7-f26d867efddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:02.466399 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec2456-52d3-4842-91d7-f26d867efddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:02.466804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec2456-52d3-4842-91d7-f26d867efddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:02.467187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ec2456-52d3-4842-91d7-f26d867efddd admin admin] PATH_INFO: `/v3/projects/cab81ac3e4864a939690848e4de734cf` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:02.468532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec2456-52d3-4842-91d7-f26d867efddd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cab81ac3e4864a939690848e4de734cf)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:02.471452 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ec2456-52d3-4842-91d7-f26d867efddd admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:02.511268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:02.522381 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-34ec2456-52d3-4842-91d7-f26d867efddd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cab81ac3e4864a939690848e4de734cf', 'request_id': 'req-34ec2456-52d3-4842-91d7-f26d867efddd'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:02.529986 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-34ec2456-52d3-4842-91d7-f26d867efddd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cab81ac3e4864a939690848e4de734cf', 'request_id': 'req-34ec2456-52d3-4842-91d7-f26d867efddd'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:02.536115 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-34ec2456-52d3-4842-91d7-f26d867efddd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cab81ac3e4864a939690848e4de734cf', 'request_id': 'req-34ec2456-52d3-4842-91d7-f26d867efddd'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:02.546912 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-34ec2456-52d3-4842-91d7-f26d867efddd admin admin] The token cache is being invalidate because project cab81ac3e4864a939690848e4de734cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:11:02.547254 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-34ec2456-52d3-4842-91d7-f26d867efddd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:02.547898 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-34ec2456-52d3-4842-91d7-f26d867efddd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:02.549367 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1103/2167] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:11:02 2022] DELETE /identity/v3/projects/cab81ac3e4864a939690848e4de734cf => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:11:02.553324 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1a5db-335d-4683-802a-cf87070b5ad1 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:02.562972 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1a5db-335d-4683-802a-cf87070b5ad1 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:02.595731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:02.624652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865fb174-c9ce-4c64-9687-3efa9b8791d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:02.625264 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fb174-c9ce-4c64-9687-3efa9b8791d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:02.625552 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fb174-c9ce-4c64-9687-3efa9b8791d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:02.625808 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fb174-c9ce-4c64-9687-3efa9b8791d7 admin admin] PATH_INFO: `/v3/users/b6b6c4910d0747b28cbcccb72c7a6d7e` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:02.626921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865fb174-c9ce-4c64-9687-3efa9b8791d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6b6c4910d0747b28cbcccb72c7a6d7e)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:02.628849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865fb174-c9ce-4c64-9687-3efa9b8791d7 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:02.681184 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-865fb174-c9ce-4c64-9687-3efa9b8791d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6b6c4910d0747b28cbcccb72c7a6d7e'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:02.683356 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-865fb174-c9ce-4c64-9687-3efa9b8791d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6b6c4910d0747b28cbcccb72c7a6d7e'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:02.701553 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-865fb174-c9ce-4c64-9687-3efa9b8791d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6b6c4910d0747b28cbcccb72c7a6d7e'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:02.726330 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-865fb174-c9ce-4c64-9687-3efa9b8791d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6b6c4910d0747b28cbcccb72c7a6d7e'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:02.727715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-865fb174-c9ce-4c64-9687-3efa9b8791d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6b6c4910d0747b28cbcccb72c7a6d7e'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:02.749127 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1065/2168] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:11:02 2022] DELETE /identity/v3/users/b6b6c4910d0747b28cbcccb72c7a6d7e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:11:03.328812 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec2456-52d3-4842-91d7-f26d867efddd admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:03.330322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-34ec2456-52d3-4842-91d7-f26d867efddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:03.391061 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ec2456-52d3-4842-91d7-f26d867efddd admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:03.591308 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed18a6-262a-4790-9a5e-e1d151bc86c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:03.591962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed18a6-262a-4790-9a5e-e1d151bc86c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:03.592273 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed18a6-262a-4790-9a5e-e1d151bc86c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:03.592853 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed18a6-262a-4790-9a5e-e1d151bc86c1 admin admin] PATH_INFO: `/v3/projects/18c6eff7f04b461d84d7d3ebc445a937` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:03.607906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed18a6-262a-4790-9a5e-e1d151bc86c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18c6eff7f04b461d84d7d3ebc445a937)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:03.610380 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed18a6-262a-4790-9a5e-e1d151bc86c1 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:03.719453 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-cfed18a6-262a-4790-9a5e-e1d151bc86c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18c6eff7f04b461d84d7d3ebc445a937', 'request_id': 'req-cfed18a6-262a-4790-9a5e-e1d151bc86c1'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:03.738664 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-cfed18a6-262a-4790-9a5e-e1d151bc86c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18c6eff7f04b461d84d7d3ebc445a937', 'request_id': 'req-cfed18a6-262a-4790-9a5e-e1d151bc86c1'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:03.760109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-cfed18a6-262a-4790-9a5e-e1d151bc86c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18c6eff7f04b461d84d7d3ebc445a937', 'request_id': 'req-cfed18a6-262a-4790-9a5e-e1d151bc86c1'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:03.777133 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-cfed18a6-262a-4790-9a5e-e1d151bc86c1 admin admin] The token cache is being invalidate because project 18c6eff7f04b461d84d7d3ebc445a937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:11:03.778318 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-cfed18a6-262a-4790-9a5e-e1d151bc86c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:03.779217 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-cfed18a6-262a-4790-9a5e-e1d151bc86c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:03.781298 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1104/2169] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:11:03 2022] DELETE /identity/v3/projects/18c6eff7f04b461d84d7d3ebc445a937 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:11:03.786356 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e4682-2066-4b03-9721-b6e58e51a7c1 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:03.787409 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e4682-2066-4b03-9721-b6e58e51a7c1 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:03.787937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e4682-2066-4b03-9721-b6e58e51a7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:03.812750 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-9e3e4682-2066-4b03-9721-b6e58e51a7c1 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:03.814966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-9e3e4682-2066-4b03-9721-b6e58e51a7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:03.903869 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1066/2170] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:11:03.912495 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f844fda-7c38-4e74-86b1-cb354528784b None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:03.912915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f844fda-7c38-4e74-86b1-cb354528784b None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:03.913348 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f844fda-7c38-4e74-86b1-cb354528784b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:03.941531 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-1f844fda-7c38-4e74-86b1-cb354528784b None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:03.943132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-1f844fda-7c38-4e74-86b1-cb354528784b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:03.952842 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1105/2171] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:11:03.957396 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e4682-2066-4b03-9721-b6e58e51a7c1 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:03.970441 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e4682-2066-4b03-9721-b6e58e51a7c1 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:03.975950 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b8b7a-681e-476b-8f62-00c6cd4d4ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:03.976559 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b8b7a-681e-476b-8f62-00c6cd4d4ddd admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:03.976904 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b8b7a-681e-476b-8f62-00c6cd4d4ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:03.977176 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b8b7a-681e-476b-8f62-00c6cd4d4ddd admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:03.977950 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b8b7a-681e-476b-8f62-00c6cd4d4ddd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:11:03.978258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b8b7a-681e-476b-8f62-00c6cd4d4ddd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:03.980185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071b8b7a-681e-476b-8f62-00c6cd4d4ddd admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:03.993984 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1067/2172] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:11:03.998568 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f844fda-7c38-4e74-86b1-cb354528784b None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:04.008826 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f844fda-7c38-4e74-86b1-cb354528784b None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:04.013825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd6522-1ed2-4a48-88a1-7995135f1594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:04.014510 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd6522-1ed2-4a48-88a1-7995135f1594 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:04.014860 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd6522-1ed2-4a48-88a1-7995135f1594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:04.015255 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd6522-1ed2-4a48-88a1-7995135f1594 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:04.016017 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd6522-1ed2-4a48-88a1-7995135f1594 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:04.018048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd6522-1ed2-4a48-88a1-7995135f1594 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:04.046147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1106/2173] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:11:03 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:11:04.050959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b8b7a-681e-476b-8f62-00c6cd4d4ddd admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:04.062072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b8b7a-681e-476b-8f62-00c6cd4d4ddd admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:04.110788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:04.132283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba0e97-1978-4adb-9cd9-7ef3581258e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:04.132888 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba0e97-1978-4adb-9cd9-7ef3581258e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:04.133165 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba0e97-1978-4adb-9cd9-7ef3581258e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:04.133435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba0e97-1978-4adb-9cd9-7ef3581258e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:04.134069 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba0e97-1978-4adb-9cd9-7ef3581258e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:04.136060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba0e97-1978-4adb-9cd9-7ef3581258e6 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:04.171387 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1068/2174] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:11:04 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:11:04.173807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd6522-1ed2-4a48-88a1-7995135f1594 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:04.185008 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd6522-1ed2-4a48-88a1-7995135f1594 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:04.190391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9adf2-9ea3-437a-a3f9-327ad4078885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:04.191029 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9adf2-9ea3-437a-a3f9-327ad4078885 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:04.191457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9adf2-9ea3-437a-a3f9-327ad4078885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:04.191696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c9adf2-9ea3-437a-a3f9-327ad4078885 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:04.192314 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c9adf2-9ea3-437a-a3f9-327ad4078885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:04.193977 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c9adf2-9ea3-437a-a3f9-327ad4078885 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:04.207214 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1107/2175] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:11:04.212116 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba0e97-1978-4adb-9cd9-7ef3581258e6 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:04.252745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba0e97-1978-4adb-9cd9-7ef3581258e6 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:04.261427 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f372a1-02a3-4824-8240-2e102d633037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:04.262391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f372a1-02a3-4824-8240-2e102d633037 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:04.262538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f372a1-02a3-4824-8240-2e102d633037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:04.262773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f372a1-02a3-4824-8240-2e102d633037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:04.263501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f372a1-02a3-4824-8240-2e102d633037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:04.266312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f372a1-02a3-4824-8240-2e102d633037 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:04.277871 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1069/2176] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:11:04.282510 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9adf2-9ea3-437a-a3f9-327ad4078885 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:04.293227 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c9adf2-9ea3-437a-a3f9-327ad4078885 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:04.299023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb053748-7d60-4d67-9ce4-bb59e7b5ea8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:04.299631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb053748-7d60-4d67-9ce4-bb59e7b5ea8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:04.299980 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb053748-7d60-4d67-9ce4-bb59e7b5ea8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:04.300322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb053748-7d60-4d67-9ce4-bb59e7b5ea8c admin admin] PATH_INFO: `/v3/projects/4e8610267201469cac8250f947460844/users/5d82bd2b117a4bfabacf7948ff96921c/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:04.313435 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:04.316754 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb053748-7d60-4d67-9ce4-bb59e7b5ea8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8610267201469cac8250f947460844, user_id=5d82bd2b117a4bfabacf7948ff96921c, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:04.320461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb053748-7d60-4d67-9ce4-bb59e7b5ea8c admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:04.341242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1108/2177] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:11:04 2022] PUT /identity/v3/projects/4e8610267201469cac8250f947460844/users/5d82bd2b117a4bfabacf7948ff96921c/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:11:04.359314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab8fae-607a-4475-b82f-c8d249df0e54 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:04.359648 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab8fae-607a-4475-b82f-c8d249df0e54 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:04.360751 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab8fae-607a-4475-b82f-c8d249df0e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:04.394306 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-d7ab8fae-607a-4475-b82f-c8d249df0e54 None None] MFA Rules not processed for user `5d82bd2b117a4bfabacf7948ff96921c`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:04.395654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-d7ab8fae-607a-4475-b82f-c8d249df0e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:04.463598 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1070/2178] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:11:04.468243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb053748-7d60-4d67-9ce4-bb59e7b5ea8c admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:04.556536 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb053748-7d60-4d67-9ce4-bb59e7b5ea8c admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:04.561489 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d3925-bcee-4b2b-9bf2-a9a92138e256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:04.562045 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d3925-bcee-4b2b-9bf2-a9a92138e256 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:04.563320 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d3925-bcee-4b2b-9bf2-a9a92138e256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:04.563887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d3925-bcee-4b2b-9bf2-a9a92138e256 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:04.564674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d3925-bcee-4b2b-9bf2-a9a92138e256 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:04.566737 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d3925-bcee-4b2b-9bf2-a9a92138e256 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:04.595601 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1109/2179] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:11:04 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:11:04.599989 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab8fae-607a-4475-b82f-c8d249df0e54 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:04.611783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab8fae-607a-4475-b82f-c8d249df0e54 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:04.681370 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f6138-b653-4b39-97f9-ece99cc00a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:04.681973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f6138-b653-4b39-97f9-ece99cc00a3c admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:04.682308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f6138-b653-4b39-97f9-ece99cc00a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:04.682974 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f6138-b653-4b39-97f9-ece99cc00a3c admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:04.684229 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f6138-b653-4b39-97f9-ece99cc00a3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:04.687128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f6138-b653-4b39-97f9-ece99cc00a3c admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:04.729795 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d3925-bcee-4b2b-9bf2-a9a92138e256 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:04.730474 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1071/2180] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:11:04 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:11:04.742959 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d3925-bcee-4b2b-9bf2-a9a92138e256 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:04.748424 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00824caa-0184-47be-b8ee-81428c82ba16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:04.749042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00824caa-0184-47be-b8ee-81428c82ba16 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:04.749334 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00824caa-0184-47be-b8ee-81428c82ba16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:04.749630 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00824caa-0184-47be-b8ee-81428c82ba16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:04.750302 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00824caa-0184-47be-b8ee-81428c82ba16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:04.752534 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00824caa-0184-47be-b8ee-81428c82ba16 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:04.774427 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1110/2181] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:11:04.775930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f6138-b653-4b39-97f9-ece99cc00a3c admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:04.788980 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f6138-b653-4b39-97f9-ece99cc00a3c admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:04.797046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc94d3-7b16-4627-a1e4-c28d5f0bc0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:04.798302 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc94d3-7b16-4627-a1e4-c28d5f0bc0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:04.798583 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc94d3-7b16-4627-a1e4-c28d5f0bc0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:04.798933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc94d3-7b16-4627-a1e4-c28d5f0bc0b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:04.799715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc94d3-7b16-4627-a1e4-c28d5f0bc0b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:04.801686 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc94d3-7b16-4627-a1e4-c28d5f0bc0b1 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:04.814005 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1072/2182] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:11:04.818684 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00824caa-0184-47be-b8ee-81428c82ba16 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:04.829845 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00824caa-0184-47be-b8ee-81428c82ba16 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:04.835709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f50610-7c66-4d67-83cf-9b7403d1a9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:04.836477 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f50610-7c66-4d67-83cf-9b7403d1a9ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:04.836758 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f50610-7c66-4d67-83cf-9b7403d1a9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:04.837031 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f50610-7c66-4d67-83cf-9b7403d1a9ad admin admin] PATH_INFO: `/v3/projects/3d269cfa98c64443a58ad34913e2b41a/users/9a7d8b3971a34d8dbda0353b07212f57/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:04.854345 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f50610-7c66-4d67-83cf-9b7403d1a9ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d269cfa98c64443a58ad34913e2b41a, user_id=9a7d8b3971a34d8dbda0353b07212f57, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:04.857183 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f50610-7c66-4d67-83cf-9b7403d1a9ad admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:04.875502 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1111/2183] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:11:04 2022] PUT /identity/v3/projects/3d269cfa98c64443a58ad34913e2b41a/users/9a7d8b3971a34d8dbda0353b07212f57/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:11:04.886508 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe731c0-462d-4412-95a8-c750b8045aec None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:04.886988 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe731c0-462d-4412-95a8-c750b8045aec None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:04.887411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe731c0-462d-4412-95a8-c750b8045aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:04.946349 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-ebe731c0-462d-4412-95a8-c750b8045aec None None] MFA Rules not processed for user `9a7d8b3971a34d8dbda0353b07212f57`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:04.948995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ebe731c0-462d-4412-95a8-c750b8045aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:05.026049 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:05.120671 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1073/2184] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:11:05.125928 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f50610-7c66-4d67-83cf-9b7403d1a9ad admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:05.214863 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f50610-7c66-4d67-83cf-9b7403d1a9ad admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:05.219856 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179caec5-b873-4df2-a713-d2190f727678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:05.220487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179caec5-b873-4df2-a713-d2190f727678 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:05.220751 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179caec5-b873-4df2-a713-d2190f727678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:05.221036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179caec5-b873-4df2-a713-d2190f727678 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:05.221966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179caec5-b873-4df2-a713-d2190f727678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:05.223968 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179caec5-b873-4df2-a713-d2190f727678 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:05.266592 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1112/2185] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:11:05 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:11:05.267587 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe731c0-462d-4412-95a8-c750b8045aec None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:05.282367 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe731c0-462d-4412-95a8-c750b8045aec None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:05.357491 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf18f3d-0581-4ec1-822e-343b392cde03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:05.358371 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf18f3d-0581-4ec1-822e-343b392cde03 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:05.358650 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf18f3d-0581-4ec1-822e-343b392cde03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:05.358999 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf18f3d-0581-4ec1-822e-343b392cde03 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:05.359857 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf18f3d-0581-4ec1-822e-343b392cde03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:05.362123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf18f3d-0581-4ec1-822e-343b392cde03 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:05.385856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1074/2186] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:11:05 2022] POST /identity/v3/users => generated 299 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:11:05.389631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179caec5-b873-4df2-a713-d2190f727678 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:05.401170 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179caec5-b873-4df2-a713-d2190f727678 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:05.406791 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b283a-8207-4ab7-8486-fb7531544c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:05.407454 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b283a-8207-4ab7-8486-fb7531544c0f admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:05.407741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b283a-8207-4ab7-8486-fb7531544c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:05.408010 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b283a-8207-4ab7-8486-fb7531544c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:05.408662 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2b283a-8207-4ab7-8486-fb7531544c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:05.410296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2b283a-8207-4ab7-8486-fb7531544c0f admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:05.424884 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1113/2187] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:11:05.426150 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf18f3d-0581-4ec1-822e-343b392cde03 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:05.438597 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf18f3d-0581-4ec1-822e-343b392cde03 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:05.444303 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6143f61-e198-436f-ba78-50c26dddaca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:05.445164 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6143f61-e198-436f-ba78-50c26dddaca7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:05.445445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6143f61-e198-436f-ba78-50c26dddaca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:05.445733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6143f61-e198-436f-ba78-50c26dddaca7 admin admin] PATH_INFO: `/v3/projects/fb2d91b090ef47fab2bf172e874acadb/users/39c21a21b62e4d87adbbf452d1643846/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:05.461865 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6143f61-e198-436f-ba78-50c26dddaca7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb2d91b090ef47fab2bf172e874acadb, user_id=39c21a21b62e4d87adbbf452d1643846, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:05.465022 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6143f61-e198-436f-ba78-50c26dddaca7 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:05.479755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1075/2188] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:11:05 2022] PUT /identity/v3/projects/fb2d91b090ef47fab2bf172e874acadb/users/39c21a21b62e4d87adbbf452d1643846/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:11:05.483852 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b283a-8207-4ab7-8486-fb7531544c0f admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:05.495395 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b283a-8207-4ab7-8486-fb7531544c0f admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:05.514234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:05.585059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41263e5-54b3-42e1-94ea-297933d0e948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:05.585813 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41263e5-54b3-42e1-94ea-297933d0e948 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:05.586366 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41263e5-54b3-42e1-94ea-297933d0e948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:05.586517 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41263e5-54b3-42e1-94ea-297933d0e948 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:05.587459 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41263e5-54b3-42e1-94ea-297933d0e948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:05.591223 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41263e5-54b3-42e1-94ea-297933d0e948 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:05.603898 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1114/2189] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:11:05.608790 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6143f61-e198-436f-ba78-50c26dddaca7 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:05.622728 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6143f61-e198-436f-ba78-50c26dddaca7 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:05.629869 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030542b-6c5a-4c13-8cfa-61a97d7337f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:05.630639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030542b-6c5a-4c13-8cfa-61a97d7337f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:05.631088 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030542b-6c5a-4c13-8cfa-61a97d7337f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:05.631618 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030542b-6c5a-4c13-8cfa-61a97d7337f9 admin admin] PATH_INFO: `/v3/projects/fb2d91b090ef47fab2bf172e874acadb/users/39c21a21b62e4d87adbbf452d1643846/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:05.633854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030542b-6c5a-4c13-8cfa-61a97d7337f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb2d91b090ef47fab2bf172e874acadb, user_id=39c21a21b62e4d87adbbf452d1643846, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:05.637538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030542b-6c5a-4c13-8cfa-61a97d7337f9 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:05.648933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1076/2190] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:11:05 2022] PUT /identity/v3/projects/fb2d91b090ef47fab2bf172e874acadb/users/39c21a21b62e4d87adbbf452d1643846/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:11:05.659681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c450d28-6689-4314-8848-1437c4b2c5ae None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:05.660043 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c450d28-6689-4314-8848-1437c4b2c5ae None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:05.660431 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c450d28-6689-4314-8848-1437c4b2c5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:05.696002 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-6c450d28-6689-4314-8848-1437c4b2c5ae None None] MFA Rules not processed for user `39c21a21b62e4d87adbbf452d1643846`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:05.697550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6c450d28-6689-4314-8848-1437c4b2c5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:05.771775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1115/2191] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:11:05.785626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030542b-6c5a-4c13-8cfa-61a97d7337f9 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:05.787550 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-e030542b-6c5a-4c13-8cfa-61a97d7337f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:05.858422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030542b-6c5a-4c13-8cfa-61a97d7337f9 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:05.863364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a195166-6b20-4767-9936-1e8081a04f87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:05.864033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a195166-6b20-4767-9936-1e8081a04f87 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:05.864315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a195166-6b20-4767-9936-1e8081a04f87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:05.864588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a195166-6b20-4767-9936-1e8081a04f87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:05.890593 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:05.930068 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a195166-6b20-4767-9936-1e8081a04f87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:05.932096 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a195166-6b20-4767-9936-1e8081a04f87 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:05.942170 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1077/2192] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:11:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:11:07.596885 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c450d28-6689-4314-8848-1437c4b2c5ae None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:07.609160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c450d28-6689-4314-8848-1437c4b2c5ae None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:07.615355 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd1cf3-b5ee-4d13-a1db-8f2aab1a8c9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:07.616338 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd1cf3-b5ee-4d13-a1db-8f2aab1a8c9d service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:07.617056 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd1cf3-b5ee-4d13-a1db-8f2aab1a8c9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:07.617520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd1cf3-b5ee-4d13-a1db-8f2aab1a8c9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:07.801008 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd1cf3-b5ee-4d13-a1db-8f2aab1a8c9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:07.805311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd1cf3-b5ee-4d13-a1db-8f2aab1a8c9d service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:07.839156 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1116/2193] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:11:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:11:08.534278 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a195166-6b20-4767-9936-1e8081a04f87 service cinder] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:08.542589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a195166-6b20-4767-9936-1e8081a04f87 service cinder] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:08.546300 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e416a7-2190-4e98-a190-557795d16264 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:08.546913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e416a7-2190-4e98-a190-557795d16264 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:08.547249 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e416a7-2190-4e98-a190-557795d16264 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:08.547559 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e416a7-2190-4e98-a190-557795d16264 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:08.558320 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e416a7-2190-4e98-a190-557795d16264 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:08.559892 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e416a7-2190-4e98-a190-557795d16264 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:08.569805 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1078/2194] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:11:08 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:11:12.452718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78454500-f5a2-4389-b7ff-e5b384a8d1eb None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:12.453233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78454500-f5a2-4389-b7ff-e5b384a8d1eb None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:12.453316 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78454500-f5a2-4389-b7ff-e5b384a8d1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:12.477236 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-78454500-f5a2-4389-b7ff-e5b384a8d1eb None None] MFA Rules not processed for user `bf58157cbc8e49c5bd913c68ff859bfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:12.478606 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-78454500-f5a2-4389-b7ff-e5b384a8d1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:12.529816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:12.563204 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1117/2195] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:11:12.836289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e416a7-2190-4e98-a190-557795d16264 service cinder] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:12.905562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e416a7-2190-4e98-a190-557795d16264 service cinder] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:12.911685 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebda35-0f14-43e1-8f7e-f4de97e92733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:12.912323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebda35-0f14-43e1-8f7e-f4de97e92733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:12.912649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebda35-0f14-43e1-8f7e-f4de97e92733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:12.912958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebda35-0f14-43e1-8f7e-f4de97e92733 admin admin] PATH_INFO: `/v3/users/5d82bd2b117a4bfabacf7948ff96921c` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:12.914619 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebda35-0f14-43e1-8f7e-f4de97e92733 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d82bd2b117a4bfabacf7948ff96921c)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:12.916963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ebda35-0f14-43e1-8f7e-f4de97e92733 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:12.968992 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-50ebda35-0f14-43e1-8f7e-f4de97e92733 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d82bd2b117a4bfabacf7948ff96921c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:12.970950 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-50ebda35-0f14-43e1-8f7e-f4de97e92733 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d82bd2b117a4bfabacf7948ff96921c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:12.985031 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-50ebda35-0f14-43e1-8f7e-f4de97e92733 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d82bd2b117a4bfabacf7948ff96921c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:12.992621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-50ebda35-0f14-43e1-8f7e-f4de97e92733 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d82bd2b117a4bfabacf7948ff96921c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:12.993302 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-50ebda35-0f14-43e1-8f7e-f4de97e92733 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d82bd2b117a4bfabacf7948ff96921c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:13.007772 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1079/2196] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:11:12 2022] DELETE /identity/v3/users/5d82bd2b117a4bfabacf7948ff96921c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:11:13.012394 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca63e83-7b15-4959-a836-68f88adcb3fc None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:13.012686 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca63e83-7b15-4959-a836-68f88adcb3fc None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:13.013142 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca63e83-7b15-4959-a836-68f88adcb3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:13.033472 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-6ca63e83-7b15-4959-a836-68f88adcb3fc None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:13.034666 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6ca63e83-7b15-4959-a836-68f88adcb3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:13.111110 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1118/2197] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:11:13.120601 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebda35-0f14-43e1-8f7e-f4de97e92733 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:13.122682 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-50ebda35-0f14-43e1-8f7e-f4de97e92733 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:13.167726 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:13.194984 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebda35-0f14-43e1-8f7e-f4de97e92733 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:13.199678 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84a9b1-5572-4e87-8e41-86dfcfc1e178 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:13.200325 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84a9b1-5572-4e87-8e41-86dfcfc1e178 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:13.200800 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84a9b1-5572-4e87-8e41-86dfcfc1e178 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:13.200917 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d84a9b1-5572-4e87-8e41-86dfcfc1e178 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:13.212491 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84a9b1-5572-4e87-8e41-86dfcfc1e178 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:13.214152 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d84a9b1-5572-4e87-8e41-86dfcfc1e178 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:13.226580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1080/2198] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:11:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:11:13.539392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca63e83-7b15-4959-a836-68f88adcb3fc None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:13.541900 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6ca63e83-7b15-4959-a836-68f88adcb3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:13.561631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca63e83-7b15-4959-a836-68f88adcb3fc None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:13.567856 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d631b8-ef7f-409c-bb11-1d5a4215efae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:13.568526 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d631b8-ef7f-409c-bb11-1d5a4215efae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:13.568940 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d631b8-ef7f-409c-bb11-1d5a4215efae admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:13.570899 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d631b8-ef7f-409c-bb11-1d5a4215efae admin admin] PATH_INFO: `/v3/projects/4e8610267201469cac8250f947460844` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:13.570899 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d631b8-ef7f-409c-bb11-1d5a4215efae admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e8610267201469cac8250f947460844)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:13.577451 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d631b8-ef7f-409c-bb11-1d5a4215efae admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:13.647716 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-91d631b8-ef7f-409c-bb11-1d5a4215efae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e8610267201469cac8250f947460844', 'request_id': 'req-91d631b8-ef7f-409c-bb11-1d5a4215efae'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:13.658220 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-91d631b8-ef7f-409c-bb11-1d5a4215efae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e8610267201469cac8250f947460844', 'request_id': 'req-91d631b8-ef7f-409c-bb11-1d5a4215efae'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:13.668353 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-91d631b8-ef7f-409c-bb11-1d5a4215efae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e8610267201469cac8250f947460844', 'request_id': 'req-91d631b8-ef7f-409c-bb11-1d5a4215efae'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:13.685995 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-91d631b8-ef7f-409c-bb11-1d5a4215efae admin admin] The token cache is being invalidate because project 4e8610267201469cac8250f947460844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:11:13.686985 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-91d631b8-ef7f-409c-bb11-1d5a4215efae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:13.688544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-91d631b8-ef7f-409c-bb11-1d5a4215efae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:13.690570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1119/2199] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:11:13 2022] DELETE /identity/v3/projects/4e8610267201469cac8250f947460844 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:11:13.695994 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84a9b1-5572-4e87-8e41-86dfcfc1e178 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:13.716936 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d84a9b1-5572-4e87-8e41-86dfcfc1e178 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:13.801995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd3f7f-1f80-41b2-a9e7-1bd852b096ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:13.803040 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd3f7f-1f80-41b2-a9e7-1bd852b096ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:13.803636 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd3f7f-1f80-41b2-a9e7-1bd852b096ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:13.804088 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd3f7f-1f80-41b2-a9e7-1bd852b096ef admin admin] PATH_INFO: `/v3/users/9a7d8b3971a34d8dbda0353b07212f57` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:13.806239 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd3f7f-1f80-41b2-a9e7-1bd852b096ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a7d8b3971a34d8dbda0353b07212f57)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:13.810052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd3f7f-1f80-41b2-a9e7-1bd852b096ef admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:13.894179 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-10fd3f7f-1f80-41b2-a9e7-1bd852b096ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a7d8b3971a34d8dbda0353b07212f57'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:13.895145 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-10fd3f7f-1f80-41b2-a9e7-1bd852b096ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a7d8b3971a34d8dbda0353b07212f57'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:13.910627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-10fd3f7f-1f80-41b2-a9e7-1bd852b096ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a7d8b3971a34d8dbda0353b07212f57'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:13.918033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-10fd3f7f-1f80-41b2-a9e7-1bd852b096ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a7d8b3971a34d8dbda0353b07212f57'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:13.918931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-10fd3f7f-1f80-41b2-a9e7-1bd852b096ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a7d8b3971a34d8dbda0353b07212f57'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:13.932909 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1081/2200] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:11:13 2022] DELETE /identity/v3/users/9a7d8b3971a34d8dbda0353b07212f57 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:11:14.228619 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d631b8-ef7f-409c-bb11-1d5a4215efae admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:14.230447 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-91d631b8-ef7f-409c-bb11-1d5a4215efae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:14.259418 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d631b8-ef7f-409c-bb11-1d5a4215efae admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:14.332976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57084697-ee5e-4712-9e66-629a009ec3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:14.333804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57084697-ee5e-4712-9e66-629a009ec3f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:14.334090 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57084697-ee5e-4712-9e66-629a009ec3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:14.334366 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57084697-ee5e-4712-9e66-629a009ec3f8 admin admin] PATH_INFO: `/v3/projects/3d269cfa98c64443a58ad34913e2b41a` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:14.335668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57084697-ee5e-4712-9e66-629a009ec3f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d269cfa98c64443a58ad34913e2b41a)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:14.338511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57084697-ee5e-4712-9e66-629a009ec3f8 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:14.345443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:14.392798 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-57084697-ee5e-4712-9e66-629a009ec3f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d269cfa98c64443a58ad34913e2b41a', 'request_id': 'req-57084697-ee5e-4712-9e66-629a009ec3f8'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:14.403441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-57084697-ee5e-4712-9e66-629a009ec3f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d269cfa98c64443a58ad34913e2b41a', 'request_id': 'req-57084697-ee5e-4712-9e66-629a009ec3f8'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:14.409796 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-57084697-ee5e-4712-9e66-629a009ec3f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d269cfa98c64443a58ad34913e2b41a', 'request_id': 'req-57084697-ee5e-4712-9e66-629a009ec3f8'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:14.421411 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-57084697-ee5e-4712-9e66-629a009ec3f8 admin admin] The token cache is being invalidate because project 3d269cfa98c64443a58ad34913e2b41a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:11:14.421615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-57084697-ee5e-4712-9e66-629a009ec3f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:14.422861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-57084697-ee5e-4712-9e66-629a009ec3f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:14.424524 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1120/2201] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:11:14 2022] DELETE /identity/v3/projects/3d269cfa98c64443a58ad34913e2b41a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:11:14.428627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd3f7f-1f80-41b2-a9e7-1bd852b096ef admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:14.430120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-10fd3f7f-1f80-41b2-a9e7-1bd852b096ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:14.447145 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd3f7f-1f80-41b2-a9e7-1bd852b096ef admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:14.544000 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:14.555301 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db08c26-8488-47ac-bc0f-5a9fbe388817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:14.555958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db08c26-8488-47ac-bc0f-5a9fbe388817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:14.556237 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db08c26-8488-47ac-bc0f-5a9fbe388817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:14.556760 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db08c26-8488-47ac-bc0f-5a9fbe388817 admin admin] PATH_INFO: `/v3/users/39c21a21b62e4d87adbbf452d1643846` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:14.557959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db08c26-8488-47ac-bc0f-5a9fbe388817 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39c21a21b62e4d87adbbf452d1643846)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:14.560103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db08c26-8488-47ac-bc0f-5a9fbe388817 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:14.662127 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9db08c26-8488-47ac-bc0f-5a9fbe388817 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39c21a21b62e4d87adbbf452d1643846'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:14.662903 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9db08c26-8488-47ac-bc0f-5a9fbe388817 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39c21a21b62e4d87adbbf452d1643846'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:14.684530 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9db08c26-8488-47ac-bc0f-5a9fbe388817 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39c21a21b62e4d87adbbf452d1643846'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:14.693452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9db08c26-8488-47ac-bc0f-5a9fbe388817 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39c21a21b62e4d87adbbf452d1643846'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:14.694368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9db08c26-8488-47ac-bc0f-5a9fbe388817 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39c21a21b62e4d87adbbf452d1643846'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:14.732120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1082/2202] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:11:14 2022] DELETE /identity/v3/users/39c21a21b62e4d87adbbf452d1643846 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:11:15.088830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57084697-ee5e-4712-9e66-629a009ec3f8 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:15.091700 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-57084697-ee5e-4712-9e66-629a009ec3f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:15.107816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57084697-ee5e-4712-9e66-629a009ec3f8 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:15.185025 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1af5d2-a702-4fae-9f44-3611994c8462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:15.185752 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1af5d2-a702-4fae-9f44-3611994c8462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:15.186186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1af5d2-a702-4fae-9f44-3611994c8462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:15.186464 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1af5d2-a702-4fae-9f44-3611994c8462 admin admin] PATH_INFO: `/v3/projects/fb2d91b090ef47fab2bf172e874acadb` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:15.188192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1af5d2-a702-4fae-9f44-3611994c8462 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb2d91b090ef47fab2bf172e874acadb)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:15.190656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1af5d2-a702-4fae-9f44-3611994c8462 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:15.248212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-bf1af5d2-a702-4fae-9f44-3611994c8462 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb2d91b090ef47fab2bf172e874acadb', 'request_id': 'req-bf1af5d2-a702-4fae-9f44-3611994c8462'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:15.258573 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-bf1af5d2-a702-4fae-9f44-3611994c8462 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb2d91b090ef47fab2bf172e874acadb', 'request_id': 'req-bf1af5d2-a702-4fae-9f44-3611994c8462'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:15.265809 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-bf1af5d2-a702-4fae-9f44-3611994c8462 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb2d91b090ef47fab2bf172e874acadb', 'request_id': 'req-bf1af5d2-a702-4fae-9f44-3611994c8462'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:15.282128 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-bf1af5d2-a702-4fae-9f44-3611994c8462 admin admin] The token cache is being invalidate because project fb2d91b090ef47fab2bf172e874acadb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:11:15.282546 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-bf1af5d2-a702-4fae-9f44-3611994c8462 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:15.283324 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-bf1af5d2-a702-4fae-9f44-3611994c8462 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:15.286046 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1121/2203] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:11:15 2022] DELETE /identity/v3/projects/fb2d91b090ef47fab2bf172e874acadb => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:11:15.291466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f78252-2582-418a-ae94-446af2b8edac None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:15.300432 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f78252-2582-418a-ae94-446af2b8edac None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:15.300784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f78252-2582-418a-ae94-446af2b8edac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:15.321596 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-b7f78252-2582-418a-ae94-446af2b8edac None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:15.322806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b7f78252-2582-418a-ae94-446af2b8edac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:15.384592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1083/2204] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:11:15.393279 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314ad5f-7398-426d-95ed-8a24905ce7d2 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:15.393659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314ad5f-7398-426d-95ed-8a24905ce7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:15.394001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314ad5f-7398-426d-95ed-8a24905ce7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:15.415708 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-3314ad5f-7398-426d-95ed-8a24905ce7d2 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:15.417010 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-3314ad5f-7398-426d-95ed-8a24905ce7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:15.425484 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1122/2205] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:11:15.429887 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f78252-2582-418a-ae94-446af2b8edac None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:15.439749 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f78252-2582-418a-ae94-446af2b8edac None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:15.444431 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576c292-d09a-480f-b00b-39ea94c984da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:15.444986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576c292-d09a-480f-b00b-39ea94c984da admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:15.445555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576c292-d09a-480f-b00b-39ea94c984da admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:15.445555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576c292-d09a-480f-b00b-39ea94c984da admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:15.446202 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576c292-d09a-480f-b00b-39ea94c984da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:11:15.446498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576c292-d09a-480f-b00b-39ea94c984da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:15.448151 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576c292-d09a-480f-b00b-39ea94c984da admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:15.464272 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1084/2206] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:11:15.465204 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314ad5f-7398-426d-95ed-8a24905ce7d2 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:15.475528 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314ad5f-7398-426d-95ed-8a24905ce7d2 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:15.480213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edf240-7074-430b-a8f4-95d6258b0f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:15.480822 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edf240-7074-430b-a8f4-95d6258b0f60 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:15.481097 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edf240-7074-430b-a8f4-95d6258b0f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:15.481361 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edf240-7074-430b-a8f4-95d6258b0f60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:15.482017 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67edf240-7074-430b-a8f4-95d6258b0f60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:15.484040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67edf240-7074-430b-a8f4-95d6258b0f60 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:15.501268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:15.520213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1123/2207] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:11:15 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:11:15.521330 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576c292-d09a-480f-b00b-39ea94c984da admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:15.531816 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576c292-d09a-480f-b00b-39ea94c984da admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:15.594341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dcae9-d594-4b13-be41-a2e60f4fa64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:15.595090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dcae9-d594-4b13-be41-a2e60f4fa64b admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:15.595424 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dcae9-d594-4b13-be41-a2e60f4fa64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:15.595669 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dcae9-d594-4b13-be41-a2e60f4fa64b admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:15.596306 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3dcae9-d594-4b13-be41-a2e60f4fa64b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:15.598252 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3dcae9-d594-4b13-be41-a2e60f4fa64b admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:15.606870 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:15.623443 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1085/2208] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:11:15 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:11:15.629358 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edf240-7074-430b-a8f4-95d6258b0f60 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:15.643585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edf240-7074-430b-a8f4-95d6258b0f60 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:15.652147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924dbf31-9320-4817-bc88-4c528422c80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:15.653235 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924dbf31-9320-4817-bc88-4c528422c80a admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:15.653526 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924dbf31-9320-4817-bc88-4c528422c80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:15.653800 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924dbf31-9320-4817-bc88-4c528422c80a admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:15.654832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924dbf31-9320-4817-bc88-4c528422c80a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:15.656724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924dbf31-9320-4817-bc88-4c528422c80a admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:15.672622 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1124/2209] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:11:15.674232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dcae9-d594-4b13-be41-a2e60f4fa64b admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:15.686822 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dcae9-d594-4b13-be41-a2e60f4fa64b admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:15.693971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76784b-5abc-43af-96a3-65fae1c53dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:15.694682 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76784b-5abc-43af-96a3-65fae1c53dca admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:15.695015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76784b-5abc-43af-96a3-65fae1c53dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:15.695295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76784b-5abc-43af-96a3-65fae1c53dca admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:15.696101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76784b-5abc-43af-96a3-65fae1c53dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:15.698619 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76784b-5abc-43af-96a3-65fae1c53dca admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:15.715268 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1086/2210] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:11:15.718082 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924dbf31-9320-4817-bc88-4c528422c80a admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:15.730837 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924dbf31-9320-4817-bc88-4c528422c80a admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:15.735802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487caba-b18a-4b5d-9df4-5f1ef880b80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:15.736454 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487caba-b18a-4b5d-9df4-5f1ef880b80b admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:15.736752 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487caba-b18a-4b5d-9df4-5f1ef880b80b admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:15.737290 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1487caba-b18a-4b5d-9df4-5f1ef880b80b admin admin] PATH_INFO: `/v3/projects/8851adb9a8a0495aa7660431a99d5eee/users/f1b7b3a759604adcb7be3fe78f998f4f/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:15.754229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487caba-b18a-4b5d-9df4-5f1ef880b80b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8851adb9a8a0495aa7660431a99d5eee, user_id=f1b7b3a759604adcb7be3fe78f998f4f, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:15.757117 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1487caba-b18a-4b5d-9df4-5f1ef880b80b admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:15.771712 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1125/2211] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:11:15 2022] PUT /identity/v3/projects/8851adb9a8a0495aa7660431a99d5eee/users/f1b7b3a759604adcb7be3fe78f998f4f/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:11:15.782199 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1640343-6b6a-4fda-ae16-1fc59eaf40fd None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:15.782596 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1640343-6b6a-4fda-ae16-1fc59eaf40fd None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:15.782885 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1640343-6b6a-4fda-ae16-1fc59eaf40fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:15.814974 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-d1640343-6b6a-4fda-ae16-1fc59eaf40fd None None] MFA Rules not processed for user `f1b7b3a759604adcb7be3fe78f998f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:15.816262 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-d1640343-6b6a-4fda-ae16-1fc59eaf40fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:15.897118 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1087/2212] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:11:15.908059 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487caba-b18a-4b5d-9df4-5f1ef880b80b admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:15.909909 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-1487caba-b18a-4b5d-9df4-5f1ef880b80b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:15.974133 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1487caba-b18a-4b5d-9df4-5f1ef880b80b admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:15.978207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684dc1ec-af2c-4aab-a9cf-4cd32ada11e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:15.978864 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684dc1ec-af2c-4aab-a9cf-4cd32ada11e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:15.979177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684dc1ec-af2c-4aab-a9cf-4cd32ada11e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:15.979492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684dc1ec-af2c-4aab-a9cf-4cd32ada11e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:15.997412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684dc1ec-af2c-4aab-a9cf-4cd32ada11e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:15.999140 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684dc1ec-af2c-4aab-a9cf-4cd32ada11e4 service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:16.010055 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1126/2213] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:11:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:11:17.476991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f822e-ce18-4b13-a7e1-13cca09fc768 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:17.477498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f822e-ce18-4b13-a7e1-13cca09fc768 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:17.477609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f822e-ce18-4b13-a7e1-13cca09fc768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:17.497386 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-016f822e-ce18-4b13-a7e1-13cca09fc768 None None] MFA Rules not processed for user `f1b7b3a759604adcb7be3fe78f998f4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:17.498608 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-016f822e-ce18-4b13-a7e1-13cca09fc768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:17.506219 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1088/2214] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:11:22.599454 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dec477-9c0b-4d6f-a352-1d709989b1f1 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:22.600027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dec477-9c0b-4d6f-a352-1d709989b1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:22.600027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dec477-9c0b-4d6f-a352-1d709989b1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:22.618852 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-81dec477-9c0b-4d6f-a352-1d709989b1f1 None None] MFA Rules not processed for user `e8b5ce2249a84c01ab57c03b656b2949`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:22.620075 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-81dec477-9c0b-4d6f-a352-1d709989b1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:22.674448 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1127/2215] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:11:27.491913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:INSERT=3,keystone:SELECT=42 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:31.878124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1348cb7-4c9b-4bb2-9bd5-92f4e0171fe7 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:31.879285 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1348cb7-4c9b-4bb2-9bd5-92f4e0171fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:31.882545 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1348cb7-4c9b-4bb2-9bd5-92f4e0171fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:31.915632 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-a1348cb7-4c9b-4bb2-9bd5-92f4e0171fe7 None None] MFA Rules not processed for user `e1daf1c8526245a9854f9b8672fa4222`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:31.917141 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-a1348cb7-4c9b-4bb2-9bd5-92f4e0171fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:31.986841 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1089/2216] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:11:32.667459 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=68,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:33.024535 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68f913-7664-425f-be7b-0416586c365d None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:33.024942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68f913-7664-425f-be7b-0416586c365d None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:33.025418 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f68f913-7664-425f-be7b-0416586c365d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:33.045733 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-5f68f913-7664-425f-be7b-0416586c365d None None] MFA Rules not processed for user `76ebecce283f491e82a961d7f739878f`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:33.047242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5f68f913-7664-425f-be7b-0416586c365d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:33.122384 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1128/2217] 10.222.0.36 () {58 vars in 967 bytes} [Fri Sep 2 09:11:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:11:33.143695 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1348cb7-4c9b-4bb2-9bd5-92f4e0171fe7 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:33.145208 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-a1348cb7-4c9b-4bb2-9bd5-92f4e0171fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:33.217144 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1348cb7-4c9b-4bb2-9bd5-92f4e0171fe7 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:33.223240 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041188d-0cdc-44cd-87e7-6105436be630 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92318948dcf1407b9fc8aa5c199787ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:33.223966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041188d-0cdc-44cd-87e7-6105436be630 service swift] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:33.224406 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041188d-0cdc-44cd-87e7-6105436be630 service swift] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:33.224781 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041188d-0cdc-44cd-87e7-6105436be630 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:33.234489 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041188d-0cdc-44cd-87e7-6105436be630 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:33.236740 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041188d-0cdc-44cd-87e7-6105436be630 service swift] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:33.246464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1090/2218] 10.222.0.36 () {60 vars in 1384 bytes} [Fri Sep 2 09:11:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:11:35.863733 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68f913-7664-425f-be7b-0416586c365d None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:35.865504 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5f68f913-7664-425f-be7b-0416586c365d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:35.967220 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f68f913-7664-425f-be7b-0416586c365d None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:35.972255 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d56c6-b0f9-4cb0-baf0-5683c672fb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:35.972979 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d56c6-b0f9-4cb0-baf0-5683c672fb9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:35.973329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d56c6-b0f9-4cb0-baf0-5683c672fb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:35.973667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d56c6-b0f9-4cb0-baf0-5683c672fb9b admin admin] PATH_INFO: `/v3/users/66133313929d4c0f98ab21b1ea336456` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:35.975638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d56c6-b0f9-4cb0-baf0-5683c672fb9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=66133313929d4c0f98ab21b1ea336456)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:35.978002 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d56c6-b0f9-4cb0-baf0-5683c672fb9b admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:36.055568 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9a1d56c6-b0f9-4cb0-baf0-5683c672fb9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66133313929d4c0f98ab21b1ea336456'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:36.067736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9a1d56c6-b0f9-4cb0-baf0-5683c672fb9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66133313929d4c0f98ab21b1ea336456'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:36.081227 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9a1d56c6-b0f9-4cb0-baf0-5683c672fb9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66133313929d4c0f98ab21b1ea336456'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:36.081852 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9a1d56c6-b0f9-4cb0-baf0-5683c672fb9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66133313929d4c0f98ab21b1ea336456'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:36.082456 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9a1d56c6-b0f9-4cb0-baf0-5683c672fb9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66133313929d4c0f98ab21b1ea336456'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:36.113798 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1129/2219] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:11:35 2022] DELETE /identity/v3/users/66133313929d4c0f98ab21b1ea336456 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:11:36.681940 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041188d-0cdc-44cd-87e7-6105436be630 service swift] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:36.696862 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041188d-0cdc-44cd-87e7-6105436be630 service swift] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:36.768234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335074d-ec9a-4deb-8374-aab02eae1f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:36.769147 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335074d-ec9a-4deb-8374-aab02eae1f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:36.769657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335074d-ec9a-4deb-8374-aab02eae1f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:36.770137 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335074d-ec9a-4deb-8374-aab02eae1f06 admin admin] PATH_INFO: `/v3/projects/d1883a19a6d24b36a063a5e673495a51` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:36.771545 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335074d-ec9a-4deb-8374-aab02eae1f06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1883a19a6d24b36a063a5e673495a51)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:36.773918 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335074d-ec9a-4deb-8374-aab02eae1f06 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:36.826811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:36.839446 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-5335074d-ec9a-4deb-8374-aab02eae1f06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1883a19a6d24b36a063a5e673495a51', 'request_id': 'req-5335074d-ec9a-4deb-8374-aab02eae1f06'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:36.855241 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-5335074d-ec9a-4deb-8374-aab02eae1f06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1883a19a6d24b36a063a5e673495a51', 'request_id': 'req-5335074d-ec9a-4deb-8374-aab02eae1f06'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:36.856834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-5335074d-ec9a-4deb-8374-aab02eae1f06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1883a19a6d24b36a063a5e673495a51', 'request_id': 'req-5335074d-ec9a-4deb-8374-aab02eae1f06'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:36.876246 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-5335074d-ec9a-4deb-8374-aab02eae1f06 admin admin] The token cache is being invalidate because project d1883a19a6d24b36a063a5e673495a51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:11:36.876854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-5335074d-ec9a-4deb-8374-aab02eae1f06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:36.877390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-5335074d-ec9a-4deb-8374-aab02eae1f06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:36.883413 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1091/2220] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:11:36 2022] DELETE /identity/v3/projects/d1883a19a6d24b36a063a5e673495a51 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:11:36.884411 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb2a6e-be5b-4989-bd35-8966bd002fd2 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:36.884893 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb2a6e-be5b-4989-bd35-8966bd002fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:36.884997 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb2a6e-be5b-4989-bd35-8966bd002fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:36.906828 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-a6eb2a6e-be5b-4989-bd35-8966bd002fd2 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:36.908705 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a6eb2a6e-be5b-4989-bd35-8966bd002fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:36.931193 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:36.983119 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1130/2221] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:11:36.991460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cce929-26e0-4ed6-b65c-c4e1e1a9428e None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:36.992420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cce929-26e0-4ed6-b65c-c4e1e1a9428e None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:36.992797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cce929-26e0-4ed6-b65c-c4e1e1a9428e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:37.017278 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-a5cce929-26e0-4ed6-b65c-c4e1e1a9428e None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:37.018746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-a5cce929-26e0-4ed6-b65c-c4e1e1a9428e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:37.027013 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1092/2222] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:11:37.031622 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb2a6e-be5b-4989-bd35-8966bd002fd2 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:37.045344 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb2a6e-be5b-4989-bd35-8966bd002fd2 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:37.050719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fac144-15d2-4570-8868-7d1137ea1824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:37.051362 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fac144-15d2-4570-8868-7d1137ea1824 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:37.051639 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fac144-15d2-4570-8868-7d1137ea1824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:37.051904 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fac144-15d2-4570-8868-7d1137ea1824 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:37.052541 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fac144-15d2-4570-8868-7d1137ea1824 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:11:37.052850 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fac144-15d2-4570-8868-7d1137ea1824 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:37.054432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fac144-15d2-4570-8868-7d1137ea1824 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:37.066420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1131/2223] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:11:37.070929 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cce929-26e0-4ed6-b65c-c4e1e1a9428e None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:37.082328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cce929-26e0-4ed6-b65c-c4e1e1a9428e None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:37.088154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a3047-e05c-4818-8f3c-b000fada82cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:37.088998 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a3047-e05c-4818-8f3c-b000fada82cc admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:37.089450 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a3047-e05c-4818-8f3c-b000fada82cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:37.089704 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a3047-e05c-4818-8f3c-b000fada82cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:37.090490 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993a3047-e05c-4818-8f3c-b000fada82cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:37.092467 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993a3047-e05c-4818-8f3c-b000fada82cc admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:37.125087 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1093/2224] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:11:37 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:11:37.129281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fac144-15d2-4570-8868-7d1137ea1824 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:37.140530 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fac144-15d2-4570-8868-7d1137ea1824 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:37.210151 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee45e86-3237-468d-abb6-40dcaebd3c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:37.211030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee45e86-3237-468d-abb6-40dcaebd3c6e admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:37.211432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee45e86-3237-468d-abb6-40dcaebd3c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:37.211772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee45e86-3237-468d-abb6-40dcaebd3c6e admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:37.212695 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee45e86-3237-468d-abb6-40dcaebd3c6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:37.214726 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee45e86-3237-468d-abb6-40dcaebd3c6e admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:37.237143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1132/2225] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:11:37 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:11:37.240796 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a3047-e05c-4818-8f3c-b000fada82cc admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:37.251945 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993a3047-e05c-4818-8f3c-b000fada82cc admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:37.256810 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51b9d4-0268-4661-ba64-a155b5107c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:37.257418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51b9d4-0268-4661-ba64-a155b5107c3b admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:37.257723 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51b9d4-0268-4661-ba64-a155b5107c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:37.257969 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51b9d4-0268-4661-ba64-a155b5107c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:37.258616 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51b9d4-0268-4661-ba64-a155b5107c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:37.260238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51b9d4-0268-4661-ba64-a155b5107c3b admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:37.270430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1094/2226] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:11:37.274436 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee45e86-3237-468d-abb6-40dcaebd3c6e admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:37.284885 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee45e86-3237-468d-abb6-40dcaebd3c6e admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:37.290619 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51c227a-3ced-404d-b51d-7d64679ae145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:37.291299 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c227a-3ced-404d-b51d-7d64679ae145 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:37.291553 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c227a-3ced-404d-b51d-7d64679ae145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:37.291905 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c227a-3ced-404d-b51d-7d64679ae145 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:37.293663 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c227a-3ced-404d-b51d-7d64679ae145 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:37.294825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51c227a-3ced-404d-b51d-7d64679ae145 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:37.306969 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1133/2227] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:11:37.311090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51b9d4-0268-4661-ba64-a155b5107c3b admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:37.322691 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51b9d4-0268-4661-ba64-a155b5107c3b admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:37.327953 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21bd6f-c169-4013-94c3-850deb775207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:37.328665 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21bd6f-c169-4013-94c3-850deb775207 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:37.328934 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21bd6f-c169-4013-94c3-850deb775207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:37.329210 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21bd6f-c169-4013-94c3-850deb775207 admin admin] PATH_INFO: `/v3/projects/aa95679d8d4442a8869f1780d80181cb/users/7f34ee9bd74c401d8349fa2df26a7ae5/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:37.353802 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21bd6f-c169-4013-94c3-850deb775207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa95679d8d4442a8869f1780d80181cb, user_id=7f34ee9bd74c401d8349fa2df26a7ae5, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:37.356775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21bd6f-c169-4013-94c3-850deb775207 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:37.372677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1095/2228] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:11:37 2022] PUT /identity/v3/projects/aa95679d8d4442a8869f1780d80181cb/users/7f34ee9bd74c401d8349fa2df26a7ae5/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:11:37.383398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90fd7a-5cc4-41a4-becf-fab357abf51b None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:37.383745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90fd7a-5cc4-41a4-becf-fab357abf51b None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:37.384020 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90fd7a-5cc4-41a4-becf-fab357abf51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:37.414834 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-2c90fd7a-5cc4-41a4-becf-fab357abf51b None None] MFA Rules not processed for user `7f34ee9bd74c401d8349fa2df26a7ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:37.416187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-2c90fd7a-5cc4-41a4-becf-fab357abf51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:37.533048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1134/2229] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:11:37.553578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21bd6f-c169-4013-94c3-850deb775207 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:37.555942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-2a21bd6f-c169-4013-94c3-850deb775207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:37.635868 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21bd6f-c169-4013-94c3-850deb775207 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:37.645978 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6fd1b-3866-49ef-a381-c34789b44878 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:37.646862 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6fd1b-3866-49ef-a381-c34789b44878 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:37.647478 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6fd1b-3866-49ef-a381-c34789b44878 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:37.648050 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6fd1b-3866-49ef-a381-c34789b44878 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:37.662228 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6fd1b-3866-49ef-a381-c34789b44878 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:37.663882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6fd1b-3866-49ef-a381-c34789b44878 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:37.685676 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1096/2230] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:11:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:11:37.712641 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90fd7a-5cc4-41a4-becf-fab357abf51b None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:37.733441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:37.777235 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90fd7a-5cc4-41a4-becf-fab357abf51b None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:37.781916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda20666-12fb-4dcc-9ad4-e0b62154acc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:37.782543 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda20666-12fb-4dcc-9ad4-e0b62154acc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:37.782827 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda20666-12fb-4dcc-9ad4-e0b62154acc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:37.783131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda20666-12fb-4dcc-9ad4-e0b62154acc8 admin admin] PATH_INFO: `/v3/users/7f34ee9bd74c401d8349fa2df26a7ae5` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:37.784262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda20666-12fb-4dcc-9ad4-e0b62154acc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f34ee9bd74c401d8349fa2df26a7ae5)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:37.786190 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda20666-12fb-4dcc-9ad4-e0b62154acc8 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:37.834977 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-bda20666-12fb-4dcc-9ad4-e0b62154acc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f34ee9bd74c401d8349fa2df26a7ae5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:37.841776 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-bda20666-12fb-4dcc-9ad4-e0b62154acc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f34ee9bd74c401d8349fa2df26a7ae5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:37.850637 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-bda20666-12fb-4dcc-9ad4-e0b62154acc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f34ee9bd74c401d8349fa2df26a7ae5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:37.851283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-bda20666-12fb-4dcc-9ad4-e0b62154acc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f34ee9bd74c401d8349fa2df26a7ae5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:37.851976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-bda20666-12fb-4dcc-9ad4-e0b62154acc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f34ee9bd74c401d8349fa2df26a7ae5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:37.879457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1135/2231] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:11:37 2022] DELETE /identity/v3/users/7f34ee9bd74c401d8349fa2df26a7ae5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:11:37.881854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5bcec-a8ec-4b9a-bffc-24bf7e4d2a45 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:37.883955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5bcec-a8ec-4b9a-bffc-24bf7e4d2a45 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:37.884562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5bcec-a8ec-4b9a-bffc-24bf7e4d2a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:37.905291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-cdf5bcec-a8ec-4b9a-bffc-24bf7e4d2a45 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:37.906836 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-cdf5bcec-a8ec-4b9a-bffc-24bf7e4d2a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:37.973939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1097/2232] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:11:37.981087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda20666-12fb-4dcc-9ad4-e0b62154acc8 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:37.983243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-bda20666-12fb-4dcc-9ad4-e0b62154acc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:38.078661 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda20666-12fb-4dcc-9ad4-e0b62154acc8 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:38.092552 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26262447-190b-47ab-b654-7bccd43a7f9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:38.093537 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26262447-190b-47ab-b654-7bccd43a7f9c service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:38.094637 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26262447-190b-47ab-b654-7bccd43a7f9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:38.095189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26262447-190b-47ab-b654-7bccd43a7f9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:38.110983 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26262447-190b-47ab-b654-7bccd43a7f9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:38.112478 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26262447-190b-47ab-b654-7bccd43a7f9c service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:38.150784 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1136/2233] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:11:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:11:38.742831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5bcec-a8ec-4b9a-bffc-24bf7e4d2a45 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:38.745368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-cdf5bcec-a8ec-4b9a-bffc-24bf7e4d2a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:38.774234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:38.778825 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5bcec-a8ec-4b9a-bffc-24bf7e4d2a45 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:38.786356 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d435f9e-97ba-4435-b9ad-ae9aa6083f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:38.787279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d435f9e-97ba-4435-b9ad-ae9aa6083f52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:38.787660 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d435f9e-97ba-4435-b9ad-ae9aa6083f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:38.788154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d435f9e-97ba-4435-b9ad-ae9aa6083f52 admin admin] PATH_INFO: `/v3/projects/aa95679d8d4442a8869f1780d80181cb` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:38.790264 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d435f9e-97ba-4435-b9ad-ae9aa6083f52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa95679d8d4442a8869f1780d80181cb)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:38.791894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d435f9e-97ba-4435-b9ad-ae9aa6083f52 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:38.842462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-0d435f9e-97ba-4435-b9ad-ae9aa6083f52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa95679d8d4442a8869f1780d80181cb', 'request_id': 'req-0d435f9e-97ba-4435-b9ad-ae9aa6083f52'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:38.849179 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-0d435f9e-97ba-4435-b9ad-ae9aa6083f52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa95679d8d4442a8869f1780d80181cb', 'request_id': 'req-0d435f9e-97ba-4435-b9ad-ae9aa6083f52'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:38.854903 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-0d435f9e-97ba-4435-b9ad-ae9aa6083f52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa95679d8d4442a8869f1780d80181cb', 'request_id': 'req-0d435f9e-97ba-4435-b9ad-ae9aa6083f52'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:38.871347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-0d435f9e-97ba-4435-b9ad-ae9aa6083f52 admin admin] The token cache is being invalidate because project aa95679d8d4442a8869f1780d80181cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:11:38.871708 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-0d435f9e-97ba-4435-b9ad-ae9aa6083f52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:38.872278 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-0d435f9e-97ba-4435-b9ad-ae9aa6083f52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:38.873970 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1098/2234] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:11:38 2022] DELETE /identity/v3/projects/aa95679d8d4442a8869f1780d80181cb => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:11:38.878686 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027ad06-e84a-4b60-b745-014639807dee None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:38.878816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027ad06-e84a-4b60-b745-014639807dee None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:38.879131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8027ad06-e84a-4b60-b745-014639807dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:38.902898 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-8027ad06-e84a-4b60-b745-014639807dee None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:38.904247 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8027ad06-e84a-4b60-b745-014639807dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:38.946548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:38.979722 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1137/2235] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:11:38.982197 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df963a-d571-422d-afd8-41ba659cf307 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:38.982489 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df963a-d571-422d-afd8-41ba659cf307 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:38.982765 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df963a-d571-422d-afd8-41ba659cf307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:39.004125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-d5df963a-d571-422d-afd8-41ba659cf307 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:39.005592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-d5df963a-d571-422d-afd8-41ba659cf307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:39.014152 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1099/2236] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:11:39.018864 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027ad06-e84a-4b60-b745-014639807dee None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:39.033314 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8027ad06-e84a-4b60-b745-014639807dee None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:39.039201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63c55c-8afd-4ce3-9d11-f37264f99ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:39.040107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63c55c-8afd-4ce3-9d11-f37264f99ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:39.040605 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63c55c-8afd-4ce3-9d11-f37264f99ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:39.040954 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63c55c-8afd-4ce3-9d11-f37264f99ab0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:39.041961 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63c55c-8afd-4ce3-9d11-f37264f99ab0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:11:39.042503 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63c55c-8afd-4ce3-9d11-f37264f99ab0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:39.044428 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63c55c-8afd-4ce3-9d11-f37264f99ab0 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:39.060572 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1138/2237] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:11:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:11:39.065081 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df963a-d571-422d-afd8-41ba659cf307 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:39.074904 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df963a-d571-422d-afd8-41ba659cf307 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:39.080452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ae988-ac1a-460c-8524-567c63a6fbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:39.081278 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ae988-ac1a-460c-8524-567c63a6fbd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:39.081750 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ae988-ac1a-460c-8524-567c63a6fbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:39.082188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ae988-ac1a-460c-8524-567c63a6fbd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:39.083014 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ae988-ac1a-460c-8524-567c63a6fbd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:39.084954 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ae988-ac1a-460c-8524-567c63a6fbd2 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:39.129400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1100/2238] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:11:39 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:11:39.134453 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63c55c-8afd-4ce3-9d11-f37264f99ab0 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:39.154474 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63c55c-8afd-4ce3-9d11-f37264f99ab0 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:39.226631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682f8cc-2719-4d07-897a-f15a6f2e5812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:39.227314 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682f8cc-2719-4d07-897a-f15a6f2e5812 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:39.227593 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682f8cc-2719-4d07-897a-f15a6f2e5812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:39.228036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682f8cc-2719-4d07-897a-f15a6f2e5812 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:39.229534 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682f8cc-2719-4d07-897a-f15a6f2e5812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:39.232440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682f8cc-2719-4d07-897a-f15a6f2e5812 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:39.261172 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1139/2239] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:11:39 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:11:39.264607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ae988-ac1a-460c-8524-567c63a6fbd2 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:39.276648 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ae988-ac1a-460c-8524-567c63a6fbd2 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:39.282771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f270a-7219-4234-90d7-39c0554e21ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:39.283408 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f270a-7219-4234-90d7-39c0554e21ba admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:39.283878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f270a-7219-4234-90d7-39c0554e21ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:39.284153 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812f270a-7219-4234-90d7-39c0554e21ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:39.284874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f270a-7219-4234-90d7-39c0554e21ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:39.286651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812f270a-7219-4234-90d7-39c0554e21ba admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:39.297838 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1101/2240] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:11:39.302370 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682f8cc-2719-4d07-897a-f15a6f2e5812 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:39.313755 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682f8cc-2719-4d07-897a-f15a6f2e5812 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:39.318521 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5b1d0-6724-44e4-be3b-ef048c5508a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:39.319156 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5b1d0-6724-44e4-be3b-ef048c5508a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:39.319438 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5b1d0-6724-44e4-be3b-ef048c5508a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:39.319710 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5b1d0-6724-44e4-be3b-ef048c5508a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:39.320348 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5b1d0-6724-44e4-be3b-ef048c5508a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:39.321949 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5b1d0-6724-44e4-be3b-ef048c5508a5 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:39.333466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1140/2241] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:11:39.337619 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f270a-7219-4234-90d7-39c0554e21ba admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:39.353693 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812f270a-7219-4234-90d7-39c0554e21ba admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:39.358680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69eeb9-0041-4a2f-b3bf-2a456c005f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:39.359382 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69eeb9-0041-4a2f-b3bf-2a456c005f84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:39.359822 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69eeb9-0041-4a2f-b3bf-2a456c005f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:39.360212 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69eeb9-0041-4a2f-b3bf-2a456c005f84 admin admin] PATH_INFO: `/v3/projects/a3c224650b7247ddbd6d9a05d770cc4c/users/fac00ae0edc84b3f999ff301b46ef302/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:39.375439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff69eeb9-0041-4a2f-b3bf-2a456c005f84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3c224650b7247ddbd6d9a05d770cc4c, user_id=fac00ae0edc84b3f999ff301b46ef302, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:39.378193 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff69eeb9-0041-4a2f-b3bf-2a456c005f84 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:39.402322 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1102/2242] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:11:39 2022] PUT /identity/v3/projects/a3c224650b7247ddbd6d9a05d770cc4c/users/fac00ae0edc84b3f999ff301b46ef302/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:11:39.404080 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c6bd1-42a2-42f1-99b2-e6ad1020619f None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:39.404537 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c6bd1-42a2-42f1-99b2-e6ad1020619f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:39.404816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7c6bd1-42a2-42f1-99b2-e6ad1020619f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:39.451593 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-be7c6bd1-42a2-42f1-99b2-e6ad1020619f None None] MFA Rules not processed for user `fac00ae0edc84b3f999ff301b46ef302`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:39.452883 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-be7c6bd1-42a2-42f1-99b2-e6ad1020619f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:39.522863 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1141/2243] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:11:39.534397 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69eeb9-0041-4a2f-b3bf-2a456c005f84 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:39.536478 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ff69eeb9-0041-4a2f-b3bf-2a456c005f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:39.599451 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69eeb9-0041-4a2f-b3bf-2a456c005f84 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:39.604020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f988fc0-f1d6-43a6-a804-dde24caaa0b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:39.604711 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f988fc0-f1d6-43a6-a804-dde24caaa0b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:39.605068 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f988fc0-f1d6-43a6-a804-dde24caaa0b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:39.605387 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f988fc0-f1d6-43a6-a804-dde24caaa0b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:39.615182 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f988fc0-f1d6-43a6-a804-dde24caaa0b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:39.616635 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f988fc0-f1d6-43a6-a804-dde24caaa0b7 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:39.626830 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1103/2244] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:11:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:11:46.536170 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c6bd1-42a2-42f1-99b2-e6ad1020619f None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:46.538144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-be7c6bd1-42a2-42f1-99b2-e6ad1020619f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:46.643833 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:46.703905 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7c6bd1-42a2-42f1-99b2-e6ad1020619f None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:46.716539 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af2fe2-0c4f-4910-ad08-f566a145d2f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a0f0fdbbd20492d8c35ec4b73404632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:46.717161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af2fe2-0c4f-4910-ad08-f566a145d2f0 service placement] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:46.717439 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af2fe2-0c4f-4910-ad08-f566a145d2f0 service placement] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:46.717713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af2fe2-0c4f-4910-ad08-f566a145d2f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:46.724411 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-97af2fe2-0c4f-4910-ad08-f566a145d2f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:46.747393 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af2fe2-0c4f-4910-ad08-f566a145d2f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:46.749019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af2fe2-0c4f-4910-ad08-f566a145d2f0 service placement] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:46.771338 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1142/2245] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:11:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:11:49.622276 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:11:50.554781 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f988fc0-f1d6-43a6-a804-dde24caaa0b7 service cinder] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:50.622835 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f988fc0-f1d6-43a6-a804-dde24caaa0b7 service cinder] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:50.628318 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a813ea4-b5a7-4882-b0ce-b5541ae0a441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:50.629051 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a813ea4-b5a7-4882-b0ce-b5541ae0a441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:50.629337 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a813ea4-b5a7-4882-b0ce-b5541ae0a441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:50.629611 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a813ea4-b5a7-4882-b0ce-b5541ae0a441 admin admin] PATH_INFO: `/v3/users/fac00ae0edc84b3f999ff301b46ef302` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:50.630879 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a813ea4-b5a7-4882-b0ce-b5541ae0a441 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fac00ae0edc84b3f999ff301b46ef302)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:50.632905 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a813ea4-b5a7-4882-b0ce-b5541ae0a441 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:50.685051 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-5a813ea4-b5a7-4882-b0ce-b5541ae0a441 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fac00ae0edc84b3f999ff301b46ef302'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:50.685733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-5a813ea4-b5a7-4882-b0ce-b5541ae0a441 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fac00ae0edc84b3f999ff301b46ef302'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:50.697811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-5a813ea4-b5a7-4882-b0ce-b5541ae0a441 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fac00ae0edc84b3f999ff301b46ef302'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:50.705322 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-5a813ea4-b5a7-4882-b0ce-b5541ae0a441 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fac00ae0edc84b3f999ff301b46ef302'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:50.706006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-5a813ea4-b5a7-4882-b0ce-b5541ae0a441 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fac00ae0edc84b3f999ff301b46ef302'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:50.720736 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1104/2246] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:11:50 2022] DELETE /identity/v3/users/fac00ae0edc84b3f999ff301b46ef302 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:11:50.725180 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdc884-9949-437e-aa37-85967018a6dc None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:50.725658 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdc884-9949-437e-aa37-85967018a6dc None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:50.725815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbdc884-9949-437e-aa37-85967018a6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:50.744826 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-1dbdc884-9949-437e-aa37-85967018a6dc None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:11:50.746029 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-1dbdc884-9949-437e-aa37-85967018a6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:50.824934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1143/2247] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:11:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:11:50.834717 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a813ea4-b5a7-4882-b0ce-b5541ae0a441 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:50.836142 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-5a813ea4-b5a7-4882-b0ce-b5541ae0a441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:50.894875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a813ea4-b5a7-4882-b0ce-b5541ae0a441 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:50.898495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8d530-2568-4024-b0df-334d7ca37e67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:50.899127 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8d530-2568-4024-b0df-334d7ca37e67 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:50.899412 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8d530-2568-4024-b0df-334d7ca37e67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:50.899788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8d530-2568-4024-b0df-334d7ca37e67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:50.910696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8d530-2568-4024-b0df-334d7ca37e67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:50.912211 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8d530-2568-4024-b0df-334d7ca37e67 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:50.924322 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1105/2248] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:11:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:11:51.432813 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbdc884-9949-437e-aa37-85967018a6dc None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:11:51.434835 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-1dbdc884-9949-437e-aa37-85967018a6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:11:51.459678 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbdc884-9949-437e-aa37-85967018a6dc None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:11:51.464962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e6e06b-5508-4289-b994-72f686ed6560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:11:51.465607 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6e06b-5508-4289-b994-72f686ed6560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:11:51.465887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6e06b-5508-4289-b994-72f686ed6560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:11:51.466165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6e06b-5508-4289-b994-72f686ed6560 admin admin] PATH_INFO: `/v3/projects/a3c224650b7247ddbd6d9a05d770cc4c` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:11:51.467339 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e6e06b-5508-4289-b994-72f686ed6560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3c224650b7247ddbd6d9a05d770cc4c)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:11:51.485140 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e6e06b-5508-4289-b994-72f686ed6560 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:11:51.547660 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-75e6e06b-5508-4289-b994-72f686ed6560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3c224650b7247ddbd6d9a05d770cc4c', 'request_id': 'req-75e6e06b-5508-4289-b994-72f686ed6560'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:51.558742 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-75e6e06b-5508-4289-b994-72f686ed6560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3c224650b7247ddbd6d9a05d770cc4c', 'request_id': 'req-75e6e06b-5508-4289-b994-72f686ed6560'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:51.569921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-75e6e06b-5508-4289-b994-72f686ed6560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3c224650b7247ddbd6d9a05d770cc4c', 'request_id': 'req-75e6e06b-5508-4289-b994-72f686ed6560'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:51.589715 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-75e6e06b-5508-4289-b994-72f686ed6560 admin admin] The token cache is being invalidate because project a3c224650b7247ddbd6d9a05d770cc4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:11:51.590023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-75e6e06b-5508-4289-b994-72f686ed6560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:51.590762 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-75e6e06b-5508-4289-b994-72f686ed6560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:11:51.597861 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1144/2249] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:11:51 2022] DELETE /identity/v3/projects/a3c224650b7247ddbd6d9a05d770cc4c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:12:00.580678 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8d530-2568-4024-b0df-334d7ca37e67 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:12:00.591289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8d530-2568-4024-b0df-334d7ca37e67 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:12:00.613876 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:12:00.618929 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fdbcf-5229-49e8-81c9-cebbca1f22b0 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:00.619948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fdbcf-5229-49e8-81c9-cebbca1f22b0 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:00.620331 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fdbcf-5229-49e8-81c9-cebbca1f22b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:00.646444 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-bd3fdbcf-5229-49e8-81c9-cebbca1f22b0 None None] MFA Rules not processed for user `76ebecce283f491e82a961d7f739878f`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:12:00.648247 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-bd3fdbcf-5229-49e8-81c9-cebbca1f22b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:00.671420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07057ec8-3e34-4665-b60c-0acb49eafcee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:12:00.672084 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07057ec8-3e34-4665-b60c-0acb49eafcee service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:00.672361 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07057ec8-3e34-4665-b60c-0acb49eafcee service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:00.672632 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07057ec8-3e34-4665-b60c-0acb49eafcee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:00.674989 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-07057ec8-3e34-4665-b60c-0acb49eafcee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:00.721637 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:12:00.734789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1145/2250] 10.222.0.36 () {58 vars in 1009 bytes} [Fri Sep 2 09:12:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:12:00.747501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07057ec8-3e34-4665-b60c-0acb49eafcee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:12:00.748985 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07057ec8-3e34-4665-b60c-0acb49eafcee service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:12:00.757330 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fdbcf-5229-49e8-81c9-cebbca1f22b0 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:12:00.758828 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-bd3fdbcf-5229-49e8-81c9-cebbca1f22b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:00.760154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1106/2251] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:12:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:12:00.830773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fdbcf-5229-49e8-81c9-cebbca1f22b0 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:12:00.836897 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bda8a1-7226-4322-828c-886e3d4cecec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92318948dcf1407b9fc8aa5c199787ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:12:00.837526 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bda8a1-7226-4322-828c-886e3d4cecec service swift] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:00.837777 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bda8a1-7226-4322-828c-886e3d4cecec service swift] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:00.838045 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bda8a1-7226-4322-828c-886e3d4cecec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:00.846540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bda8a1-7226-4322-828c-886e3d4cecec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:12:00.848238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bda8a1-7226-4322-828c-886e3d4cecec service swift] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:12:00.856515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1146/2252] 10.222.0.36 () {60 vars in 1384 bytes} [Fri Sep 2 09:12:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 02 09:12:01.705111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08739b03-fb83-48ba-aed8-033242672f40 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:01.705985 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08739b03-fb83-48ba-aed8-033242672f40 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:01.706418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08739b03-fb83-48ba-aed8-033242672f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:01.748388 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-08739b03-fb83-48ba-aed8-033242672f40 None None] MFA Rules not processed for user `76ebecce283f491e82a961d7f739878f`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:12:01.749615 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-08739b03-fb83-48ba-aed8-033242672f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:01.758301 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1107/2253] 10.222.0.36 () {58 vars in 1009 bytes} [Fri Sep 2 09:12:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:12:01.778001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bda8a1-7226-4322-828c-886e3d4cecec service swift] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:12:01.789409 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bda8a1-7226-4322-828c-886e3d4cecec service swift] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:12:01.806628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4eb819-23ff-40dd-b68a-45ebde8eaa46 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92318948dcf1407b9fc8aa5c199787ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:12:01.807282 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4eb819-23ff-40dd-b68a-45ebde8eaa46 service swift] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:01.807573 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4eb819-23ff-40dd-b68a-45ebde8eaa46 service swift] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:01.807830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4eb819-23ff-40dd-b68a-45ebde8eaa46 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:01.817480 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4eb819-23ff-40dd-b68a-45ebde8eaa46 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:12:01.819053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4eb819-23ff-40dd-b68a-45ebde8eaa46 service swift] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:12:01.829057 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1147/2254] 10.222.0.36 () {60 vars in 1384 bytes} [Fri Sep 2 09:12:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:12:02.566674 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcea9a9-f62b-4b70-80bb-c4310fa16137 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:02.567111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcea9a9-f62b-4b70-80bb-c4310fa16137 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:02.567450 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcea9a9-f62b-4b70-80bb-c4310fa16137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:02.592611 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-1bcea9a9-f62b-4b70-80bb-c4310fa16137 None None] MFA Rules not processed for user `76ebecce283f491e82a961d7f739878f`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:12:02.594068 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1bcea9a9-f62b-4b70-80bb-c4310fa16137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:02.608008 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1108/2255] 10.222.0.36 () {58 vars in 967 bytes} [Fri Sep 2 09:12:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:12:02.618168 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4eb819-23ff-40dd-b68a-45ebde8eaa46 service swift] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:12:02.655369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4eb819-23ff-40dd-b68a-45ebde8eaa46 service swift] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:12:02.671785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f47bad-67b4-4ad3-91c0-1f085d8172a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92318948dcf1407b9fc8aa5c199787ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:12:02.673435 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f47bad-67b4-4ad3-91c0-1f085d8172a3 service swift] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:02.678248 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f47bad-67b4-4ad3-91c0-1f085d8172a3 service swift] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:02.678726 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f47bad-67b4-4ad3-91c0-1f085d8172a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:02.699870 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f47bad-67b4-4ad3-91c0-1f085d8172a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:12:02.705893 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f47bad-67b4-4ad3-91c0-1f085d8172a3 service swift] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:12:02.738943 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1148/2256] 10.222.0.36 () {60 vars in 1384 bytes} [Fri Sep 2 09:12:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:12:03.198368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600f90e-2cd3-4766-b7a7-337eaa59e817 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:03.198931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600f90e-2cd3-4766-b7a7-337eaa59e817 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:03.198931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600f90e-2cd3-4766-b7a7-337eaa59e817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:03.240642 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-9600f90e-2cd3-4766-b7a7-337eaa59e817 None None] MFA Rules not processed for user `76ebecce283f491e82a961d7f739878f`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:12:03.241990 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-9600f90e-2cd3-4766-b7a7-337eaa59e817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:03.265366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1109/2257] 10.222.0.36 () {58 vars in 1009 bytes} [Fri Sep 2 09:12:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:12:03.299730 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f47bad-67b4-4ad3-91c0-1f085d8172a3 service swift] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:12:03.317827 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f47bad-67b4-4ad3-91c0-1f085d8172a3 service swift] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:12:03.330998 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a1707-a63d-4c57-bb83-32dd34e9abf9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92318948dcf1407b9fc8aa5c199787ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:12:03.330998 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a1707-a63d-4c57-bb83-32dd34e9abf9 service swift] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:03.330998 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a1707-a63d-4c57-bb83-32dd34e9abf9 service swift] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:03.331618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a1707-a63d-4c57-bb83-32dd34e9abf9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:03.358225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a1707-a63d-4c57-bb83-32dd34e9abf9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:12:03.359941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a1707-a63d-4c57-bb83-32dd34e9abf9 service swift] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:12:03.375343 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1149/2258] 10.222.0.36 () {60 vars in 1384 bytes} [Fri Sep 2 09:12:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:12:03.863907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190714dc-084b-4c65-a7b7-8d06d4767dfb None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:03.864239 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190714dc-084b-4c65-a7b7-8d06d4767dfb None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:03.864520 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190714dc-084b-4c65-a7b7-8d06d4767dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:03.892360 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-190714dc-084b-4c65-a7b7-8d06d4767dfb None None] MFA Rules not processed for user `76ebecce283f491e82a961d7f739878f`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:12:03.895480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-190714dc-084b-4c65-a7b7-8d06d4767dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:03.912346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1110/2259] 10.222.0.36 () {58 vars in 967 bytes} [Fri Sep 2 09:12:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:12:03.952309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a1707-a63d-4c57-bb83-32dd34e9abf9 service swift] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:12:03.965229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a1707-a63d-4c57-bb83-32dd34e9abf9 service swift] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:12:03.971496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5705dd2a-d069-4636-897f-bfe7b67dd44d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92318948dcf1407b9fc8aa5c199787ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:12:03.972310 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705dd2a-d069-4636-897f-bfe7b67dd44d service swift] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:03.972822 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705dd2a-d069-4636-897f-bfe7b67dd44d service swift] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:03.973298 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5705dd2a-d069-4636-897f-bfe7b67dd44d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:04.028858 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705dd2a-d069-4636-897f-bfe7b67dd44d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:12:04.030525 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5705dd2a-d069-4636-897f-bfe7b67dd44d service swift] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:12:04.042526 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1150/2260] 10.222.0.36 () {60 vars in 1384 bytes} [Fri Sep 2 09:12:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:12:06.493666 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190714dc-084b-4c65-a7b7-8d06d4767dfb None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:12:06.495304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-190714dc-084b-4c65-a7b7-8d06d4767dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:06.580369 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190714dc-084b-4c65-a7b7-8d06d4767dfb None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:12:06.585008 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677fbef-1dac-427d-8e7d-59b6b69c216a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:12:06.585662 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677fbef-1dac-427d-8e7d-59b6b69c216a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:06.586057 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677fbef-1dac-427d-8e7d-59b6b69c216a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:06.586474 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677fbef-1dac-427d-8e7d-59b6b69c216a admin admin] PATH_INFO: `/v3/users/f1b7b3a759604adcb7be3fe78f998f4f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:06.587790 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677fbef-1dac-427d-8e7d-59b6b69c216a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1b7b3a759604adcb7be3fe78f998f4f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:12:06.592329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677fbef-1dac-427d-8e7d-59b6b69c216a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:12:06.663503 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-e677fbef-1dac-427d-8e7d-59b6b69c216a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1b7b3a759604adcb7be3fe78f998f4f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:12:06.665122 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-e677fbef-1dac-427d-8e7d-59b6b69c216a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1b7b3a759604adcb7be3fe78f998f4f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:12:06.675959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:12:06.687634 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-e677fbef-1dac-427d-8e7d-59b6b69c216a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1b7b3a759604adcb7be3fe78f998f4f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:12:06.695465 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-e677fbef-1dac-427d-8e7d-59b6b69c216a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1b7b3a759604adcb7be3fe78f998f4f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:12:06.696337 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-e677fbef-1dac-427d-8e7d-59b6b69c216a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1b7b3a759604adcb7be3fe78f998f4f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:12:06.710583 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1111/2261] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:12:06 2022] DELETE /identity/v3/users/f1b7b3a759604adcb7be3fe78f998f4f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:12:06.715300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ee48b-76b4-491b-8c6e-df97ffc91964 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:06.715835 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ee48b-76b4-491b-8c6e-df97ffc91964 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:06.716097 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ee48b-76b4-491b-8c6e-df97ffc91964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:06.736421 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-137ee48b-76b4-491b-8c6e-df97ffc91964 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:12:06.738397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-137ee48b-76b4-491b-8c6e-df97ffc91964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:06.805502 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1151/2262] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:12:06.816609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677fbef-1dac-427d-8e7d-59b6b69c216a admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:12:06.818517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-e677fbef-1dac-427d-8e7d-59b6b69c216a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:06.885370 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677fbef-1dac-427d-8e7d-59b6b69c216a admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:12:06.889374 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bbfd2d-621f-4d55-828f-47e863efacf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:12:06.890018 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bbfd2d-621f-4d55-828f-47e863efacf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:06.890300 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bbfd2d-621f-4d55-828f-47e863efacf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:06.890573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bbfd2d-621f-4d55-828f-47e863efacf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:06.901329 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bbfd2d-621f-4d55-828f-47e863efacf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:12:06.902962 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bbfd2d-621f-4d55-828f-47e863efacf2 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:12:06.913372 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1112/2263] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:12:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:12:07.097608 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ee48b-76b4-491b-8c6e-df97ffc91964 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:12:07.099632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-137ee48b-76b4-491b-8c6e-df97ffc91964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:07.114360 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ee48b-76b4-491b-8c6e-df97ffc91964 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:12:07.137487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef78c93-3aa2-4655-99d5-3bb183c30f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:12:07.139491 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef78c93-3aa2-4655-99d5-3bb183c30f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:07.139765 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef78c93-3aa2-4655-99d5-3bb183c30f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:07.140040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef78c93-3aa2-4655-99d5-3bb183c30f03 admin admin] PATH_INFO: `/v3/projects/8851adb9a8a0495aa7660431a99d5eee` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:07.142246 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef78c93-3aa2-4655-99d5-3bb183c30f03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8851adb9a8a0495aa7660431a99d5eee)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:12:07.144228 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef78c93-3aa2-4655-99d5-3bb183c30f03 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:12:07.184436 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:12:07.227834 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-1ef78c93-3aa2-4655-99d5-3bb183c30f03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8851adb9a8a0495aa7660431a99d5eee', 'request_id': 'req-1ef78c93-3aa2-4655-99d5-3bb183c30f03'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:12:07.239479 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-1ef78c93-3aa2-4655-99d5-3bb183c30f03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8851adb9a8a0495aa7660431a99d5eee', 'request_id': 'req-1ef78c93-3aa2-4655-99d5-3bb183c30f03'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:12:07.246799 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-1ef78c93-3aa2-4655-99d5-3bb183c30f03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8851adb9a8a0495aa7660431a99d5eee', 'request_id': 'req-1ef78c93-3aa2-4655-99d5-3bb183c30f03'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:12:07.288020 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-1ef78c93-3aa2-4655-99d5-3bb183c30f03 admin admin] The token cache is being invalidate because project 8851adb9a8a0495aa7660431a99d5eee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:12:07.288401 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-1ef78c93-3aa2-4655-99d5-3bb183c30f03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:12:07.289137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-1ef78c93-3aa2-4655-99d5-3bb183c30f03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:12:07.291159 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1152/2264] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:12:07 2022] DELETE /identity/v3/projects/8851adb9a8a0495aa7660431a99d5eee => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:12:16.909008 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=38,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:12:17.283462 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=14,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:12:19.069984 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf8a37-81bd-47e3-91b9-4312ff3f1ac3 None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:19.070549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf8a37-81bd-47e3-91b9-4312ff3f1ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:19.070549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf8a37-81bd-47e3-91b9-4312ff3f1ac3 None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:19.071892 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1113/2265] 10.222.0.36 () {54 vars in 921 bytes} [Fri Sep 2 09:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:12:19.078329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec6f9f-790d-45f2-9f4d-a41ace73e790 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:19.078828 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec6f9f-790d-45f2-9f4d-a41ace73e790 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:19.079012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec6f9f-790d-45f2-9f4d-a41ace73e790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:19.099334 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-f0ec6f9f-790d-45f2-9f4d-a41ace73e790 None None] MFA Rules not processed for user `cce502a83f5343558c2136c0193e7d85`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:12:19.100724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f0ec6f9f-790d-45f2-9f4d-a41ace73e790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:19.168030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1153/2266] 10.222.0.36 () {58 vars in 1018 bytes} [Fri Sep 2 09:12:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:12:19.179734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf8a37-81bd-47e3-91b9-4312ff3f1ac3 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:12:19.181571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-00cf8a37-81bd-47e3-91b9-4312ff3f1ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:19.279663 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf8a37-81bd-47e3-91b9-4312ff3f1ac3 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:12:19.283800 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f543515-e20d-4c92-a9ba-64aa1405873f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2474d89f10764afbbc5e740dccaae4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:12:19.284403 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f543515-e20d-4c92-a9ba-64aa1405873f service glance] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:19.284680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f543515-e20d-4c92-a9ba-64aa1405873f service glance] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:19.284952 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f543515-e20d-4c92-a9ba-64aa1405873f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:19.295641 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f543515-e20d-4c92-a9ba-64aa1405873f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:12:19.297618 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f543515-e20d-4c92-a9ba-64aa1405873f service glance] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:12:19.308521 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1114/2267] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:12:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:12:19.388956 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57278da-fb49-4a43-b673-75279fc98097 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:19.389151 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57278da-fb49-4a43-b673-75279fc98097 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:19.389438 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57278da-fb49-4a43-b673-75279fc98097 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:19.391496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1154/2268] 10.222.0.36 () {54 vars in 921 bytes} [Fri Sep 2 09:12:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:12:19.396130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629999ad-df83-4d8f-92a2-3e6b31f383c5 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:19.396462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629999ad-df83-4d8f-92a2-3e6b31f383c5 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:19.396777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629999ad-df83-4d8f-92a2-3e6b31f383c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:19.418132 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-629999ad-df83-4d8f-92a2-3e6b31f383c5 None None] MFA Rules not processed for user `07a825fddbc04f918bcd631976038cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:12:19.419397 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-629999ad-df83-4d8f-92a2-3e6b31f383c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:19.485360 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1115/2269] 10.222.0.36 () {58 vars in 1018 bytes} [Fri Sep 2 09:12:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:12:19.501027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57278da-fb49-4a43-b673-75279fc98097 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:12:19.503971 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f57278da-fb49-4a43-b673-75279fc98097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:19.521140 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57278da-fb49-4a43-b673-75279fc98097 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:12:19.526529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01084bfe-2d15-45b1-a4ea-36dbb0ffad9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:12:19.527515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01084bfe-2d15-45b1-a4ea-36dbb0ffad9d service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:19.527960 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01084bfe-2d15-45b1-a4ea-36dbb0ffad9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:19.528231 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01084bfe-2d15-45b1-a4ea-36dbb0ffad9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:19.538618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01084bfe-2d15-45b1-a4ea-36dbb0ffad9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:12:19.541098 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01084bfe-2d15-45b1-a4ea-36dbb0ffad9d service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:12:19.550817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1155/2270] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:12:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:12:19.639039 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9201410-d1de-42f9-b3b0-f82aa725db9e None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:19.639322 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9201410-d1de-42f9-b3b0-f82aa725db9e None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:19.639599 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9201410-d1de-42f9-b3b0-f82aa725db9e None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:19.641478 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1116/2271] 10.222.0.36 () {54 vars in 921 bytes} [Fri Sep 2 09:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:12:19.646736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea22fa6-865d-4836-8da2-c55fe4723c41 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:19.647021 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea22fa6-865d-4836-8da2-c55fe4723c41 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:19.647321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea22fa6-865d-4836-8da2-c55fe4723c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:19.667420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-cea22fa6-865d-4836-8da2-c55fe4723c41 None None] MFA Rules not processed for user `cce502a83f5343558c2136c0193e7d85`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:12:19.668590 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-cea22fa6-865d-4836-8da2-c55fe4723c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:19.677584 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1156/2272] 10.222.0.36 () {58 vars in 1018 bytes} [Fri Sep 2 09:12:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:12:19.688223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9201410-d1de-42f9-b3b0-f82aa725db9e None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:12:19.689896 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b9201410-d1de-42f9-b3b0-f82aa725db9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:19.703681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9201410-d1de-42f9-b3b0-f82aa725db9e None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:12:19.707248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d1e2c-c051-4cc4-ac52-99fe95ac0d6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:12:19.707838 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d1e2c-c051-4cc4-ac52-99fe95ac0d6e service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:19.708118 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d1e2c-c051-4cc4-ac52-99fe95ac0d6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:19.708390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d1e2c-c051-4cc4-ac52-99fe95ac0d6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:19.718328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d1e2c-c051-4cc4-ac52-99fe95ac0d6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:12:19.720389 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d1e2c-c051-4cc4-ac52-99fe95ac0d6e service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:12:19.730903 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1117/2273] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:12:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:12:25.938374 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f0068-9aae-40a2-9a44-eed453e7d0b9 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:25.938835 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f0068-9aae-40a2-9a44-eed453e7d0b9 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:25.938945 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936f0068-9aae-40a2-9a44-eed453e7d0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:25.957873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-936f0068-9aae-40a2-9a44-eed453e7d0b9 None None] MFA Rules not processed for user `76ebecce283f491e82a961d7f739878f`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:12:25.958971 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-936f0068-9aae-40a2-9a44-eed453e7d0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:26.011192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1157/2274] 10.222.0.36 () {58 vars in 1009 bytes} [Fri Sep 2 09:12:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:12:26.022969 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d1e2c-c051-4cc4-ac52-99fe95ac0d6e service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:12:26.024391 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c79d1e2c-c051-4cc4-ac52-99fe95ac0d6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:26.091679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d1e2c-c051-4cc4-ac52-99fe95ac0d6e service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:12:26.097079 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15658b76-1184-4d6e-8400-e8abfb869005 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92318948dcf1407b9fc8aa5c199787ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:12:26.097668 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15658b76-1184-4d6e-8400-e8abfb869005 service swift] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:26.097943 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15658b76-1184-4d6e-8400-e8abfb869005 service swift] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:26.098210 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15658b76-1184-4d6e-8400-e8abfb869005 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:26.105902 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15658b76-1184-4d6e-8400-e8abfb869005 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:12:26.107495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15658b76-1184-4d6e-8400-e8abfb869005 service swift] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:12:26.115838 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1118/2275] 10.222.0.36 () {60 vars in 1384 bytes} [Fri Sep 2 09:12:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:12:36.004808 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=65 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:12:36.112721 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=96 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:12:50.730141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f0068-9aae-40a2-9a44-eed453e7d0b9 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:12:50.732312 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-936f0068-9aae-40a2-9a44-eed453e7d0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:50.809728 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936f0068-9aae-40a2-9a44-eed453e7d0b9 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:12:50.814819 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0670417-3f43-4219-84c9-c23df8033a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:12:50.815483 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0670417-3f43-4219-84c9-c23df8033a59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:50.815763 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0670417-3f43-4219-84c9-c23df8033a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:50.816162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0670417-3f43-4219-84c9-c23df8033a59 admin admin] PATH_INFO: `/v3/users/e1daf1c8526245a9854f9b8672fa4222` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:50.817309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0670417-3f43-4219-84c9-c23df8033a59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1daf1c8526245a9854f9b8672fa4222)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:12:50.819627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0670417-3f43-4219-84c9-c23df8033a59 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:12:50.879132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f0670417-3f43-4219-84c9-c23df8033a59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1daf1c8526245a9854f9b8672fa4222'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:12:50.886258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f0670417-3f43-4219-84c9-c23df8033a59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1daf1c8526245a9854f9b8672fa4222'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:12:50.894702 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f0670417-3f43-4219-84c9-c23df8033a59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1daf1c8526245a9854f9b8672fa4222'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:12:50.895691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f0670417-3f43-4219-84c9-c23df8033a59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1daf1c8526245a9854f9b8672fa4222'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:12:50.896384 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f0670417-3f43-4219-84c9-c23df8033a59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1daf1c8526245a9854f9b8672fa4222'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:12:50.913419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1158/2276] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:12:50 2022] DELETE /identity/v3/users/e1daf1c8526245a9854f9b8672fa4222 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:12:51.123619 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15658b76-1184-4d6e-8400-e8abfb869005 service swift] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:12:51.135362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15658b76-1184-4d6e-8400-e8abfb869005 service swift] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:12:51.192699 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac852d-1fc9-43c1-b9e0-3144a67d23e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:12:51.193460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac852d-1fc9-43c1-b9e0-3144a67d23e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:51.193735 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac852d-1fc9-43c1-b9e0-3144a67d23e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:51.194004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac852d-1fc9-43c1-b9e0-3144a67d23e3 admin admin] PATH_INFO: `/v3/projects/85a7c3487e204650adc69b55ef3feb66` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:51.195187 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac852d-1fc9-43c1-b9e0-3144a67d23e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85a7c3487e204650adc69b55ef3feb66)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:12:51.197118 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac852d-1fc9-43c1-b9e0-3144a67d23e3 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:12:51.252256 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-70ac852d-1fc9-43c1-b9e0-3144a67d23e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85a7c3487e204650adc69b55ef3feb66', 'request_id': 'req-70ac852d-1fc9-43c1-b9e0-3144a67d23e3'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:12:51.258255 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-70ac852d-1fc9-43c1-b9e0-3144a67d23e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85a7c3487e204650adc69b55ef3feb66', 'request_id': 'req-70ac852d-1fc9-43c1-b9e0-3144a67d23e3'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:12:51.263703 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-70ac852d-1fc9-43c1-b9e0-3144a67d23e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85a7c3487e204650adc69b55ef3feb66', 'request_id': 'req-70ac852d-1fc9-43c1-b9e0-3144a67d23e3'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:12:51.276554 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-70ac852d-1fc9-43c1-b9e0-3144a67d23e3 admin admin] The token cache is being invalidate because project 85a7c3487e204650adc69b55ef3feb66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:12:51.276885 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-70ac852d-1fc9-43c1-b9e0-3144a67d23e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:12:51.277533 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-70ac852d-1fc9-43c1-b9e0-3144a67d23e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:12:51.279043 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1119/2277] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:12:51 2022] DELETE /identity/v3/projects/85a7c3487e204650adc69b55ef3feb66 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:12:54.742315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ec299-5adb-4c28-aed4-def18f915f0d None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:54.742850 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ec299-5adb-4c28-aed4-def18f915f0d None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:54.743011 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ec299-5adb-4c28-aed4-def18f915f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:54.763830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-da4ec299-5adb-4c28-aed4-def18f915f0d None None] MFA Rules not processed for user `76ebecce283f491e82a961d7f739878f`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:12:54.765750 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-da4ec299-5adb-4c28-aed4-def18f915f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:54.827646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1159/2278] 10.222.0.36 () {58 vars in 967 bytes} [Fri Sep 2 09:12:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:12:54.839973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac852d-1fc9-43c1-b9e0-3144a67d23e3 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:12:54.841797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-70ac852d-1fc9-43c1-b9e0-3144a67d23e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:54.914274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac852d-1fc9-43c1-b9e0-3144a67d23e3 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:12:54.919372 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278c8fd-2ecb-49dd-be0f-4345b77f2272 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92318948dcf1407b9fc8aa5c199787ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:12:54.919982 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278c8fd-2ecb-49dd-be0f-4345b77f2272 service swift] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:54.920261 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278c8fd-2ecb-49dd-be0f-4345b77f2272 service swift] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:54.920536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278c8fd-2ecb-49dd-be0f-4345b77f2272 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:54.929243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278c8fd-2ecb-49dd-be0f-4345b77f2272 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:12:54.930831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278c8fd-2ecb-49dd-be0f-4345b77f2272 service swift] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:12:54.939443 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1120/2279] 10.222.0.36 () {60 vars in 1384 bytes} [Fri Sep 2 09:12:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:12:58.508332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964e511-1c68-4bf6-a58e-3c46e646ae12 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:58.508811 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964e511-1c68-4bf6-a58e-3c46e646ae12 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:58.508930 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964e511-1c68-4bf6-a58e-3c46e646ae12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:58.527438 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-6964e511-1c68-4bf6-a58e-3c46e646ae12 None None] MFA Rules not processed for user `e8b5ce2249a84c01ab57c03b656b2949`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:12:58.528554 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6964e511-1c68-4bf6-a58e-3c46e646ae12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:58.546673 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:12:58.586281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1160/2280] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:12:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:13:04.935888 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:08.579576 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=14 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:24.603913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004828c-ee81-4b10-ad10-02fe40e78456 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:24.604464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004828c-ee81-4b10-ad10-02fe40e78456 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:24.604464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004828c-ee81-4b10-ad10-02fe40e78456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:24.623712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-b004828c-ee81-4b10-ad10-02fe40e78456 None None] MFA Rules not processed for user `bf58157cbc8e49c5bd913c68ff859bfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:24.624942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b004828c-ee81-4b10-ad10-02fe40e78456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:24.680940 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1121/2281] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:13:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:13:34.674182 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=25 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:35.169273 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964e511-1c68-4bf6-a58e-3c46e646ae12 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:35.170777 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6964e511-1c68-4bf6-a58e-3c46e646ae12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:35.226609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964e511-1c68-4bf6-a58e-3c46e646ae12 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:35.230164 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79726a1-5c25-43b4-86aa-2aaecdd9320f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:35.230757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79726a1-5c25-43b4-86aa-2aaecdd9320f service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:35.231032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79726a1-5c25-43b4-86aa-2aaecdd9320f service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:35.231333 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79726a1-5c25-43b4-86aa-2aaecdd9320f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:35.233012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c79726a1-5c25-43b4-86aa-2aaecdd9320f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:35.302846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79726a1-5c25-43b4-86aa-2aaecdd9320f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:35.304244 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79726a1-5c25-43b4-86aa-2aaecdd9320f service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:35.315716 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1161/2282] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:13:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:13:37.949723 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004828c-ee81-4b10-ad10-02fe40e78456 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:37.951185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b004828c-ee81-4b10-ad10-02fe40e78456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:38.007884 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004828c-ee81-4b10-ad10-02fe40e78456 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:38.011458 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903353b-94a5-4369-bdc1-b730e0d489b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:38.012066 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903353b-94a5-4369-bdc1-b730e0d489b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:38.012346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903353b-94a5-4369-bdc1-b730e0d489b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:38.012619 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903353b-94a5-4369-bdc1-b730e0d489b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:38.014365 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-f903353b-94a5-4369-bdc1-b730e0d489b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:38.027966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903353b-94a5-4369-bdc1-b730e0d489b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:38.029354 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903353b-94a5-4369-bdc1-b730e0d489b5 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:38.039874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1122/2283] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:13:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:13:42.307698 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01522f39-bafd-4ae2-9d2d-8e235ea6358d None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:42.308239 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01522f39-bafd-4ae2-9d2d-8e235ea6358d None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:42.308239 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01522f39-bafd-4ae2-9d2d-8e235ea6358d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:42.327823 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-01522f39-bafd-4ae2-9d2d-8e235ea6358d None None] MFA Rules not processed for user `10fcb673c54e484a88a0dabe7b8edb15`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:42.329265 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-01522f39-bafd-4ae2-9d2d-8e235ea6358d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:42.384204 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1162/2284] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:13:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:13:48.035319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=26 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:48.889034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903353b-94a5-4369-bdc1-b730e0d489b5 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:48.890633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-f903353b-94a5-4369-bdc1-b730e0d489b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:48.908666 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903353b-94a5-4369-bdc1-b730e0d489b5 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:48.915986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b503a8e-bd7c-42c7-ab90-990487a916dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:48.916809 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b503a8e-bd7c-42c7-ab90-990487a916dd service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:48.917102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b503a8e-bd7c-42c7-ab90-990487a916dd service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:48.918029 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b503a8e-bd7c-42c7-ab90-990487a916dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:48.919914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-0b503a8e-bd7c-42c7-ab90-990487a916dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:48.933363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b503a8e-bd7c-42c7-ab90-990487a916dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:48.935016 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b503a8e-bd7c-42c7-ab90-990487a916dd service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:48.946062 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1123/2285] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:13:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:13:50.344143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01522f39-bafd-4ae2-9d2d-8e235ea6358d None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:50.371495 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01522f39-bafd-4ae2-9d2d-8e235ea6358d None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:50.386697 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490e69b9-49ce-43ee-9bdd-883a38eeb46b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:50.388817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e69b9-49ce-43ee-9bdd-883a38eeb46b service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:50.389111 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e69b9-49ce-43ee-9bdd-883a38eeb46b service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:50.389435 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490e69b9-49ce-43ee-9bdd-883a38eeb46b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:50.391683 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-490e69b9-49ce-43ee-9bdd-883a38eeb46b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:50.455226 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:50.471605 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490e69b9-49ce-43ee-9bdd-883a38eeb46b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:50.473543 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490e69b9-49ce-43ee-9bdd-883a38eeb46b service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:50.483956 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1163/2286] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:13:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:13:54.663380 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbaf6b-951c-4089-81c4-8561ece3b044 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:54.663902 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbaf6b-951c-4089-81c4-8561ece3b044 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:54.664016 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbaf6b-951c-4089-81c4-8561ece3b044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:54.682512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-6ebbaf6b-951c-4089-81c4-8561ece3b044 None None] MFA Rules not processed for user `4210f948dc7e4a5bac61d10eb6a62eee`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:54.683736 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6ebbaf6b-951c-4089-81c4-8561ece3b044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:54.690979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1124/2287] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:13:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:13:54.987255 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490e69b9-49ce-43ee-9bdd-883a38eeb46b service nova] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:54.996513 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490e69b9-49ce-43ee-9bdd-883a38eeb46b service nova] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:55.000269 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d77b74-d02f-4772-ab97-f9ed4c0fabf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:55.000863 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d77b74-d02f-4772-ab97-f9ed4c0fabf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:55.001199 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d77b74-d02f-4772-ab97-f9ed4c0fabf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:55.001489 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d77b74-d02f-4772-ab97-f9ed4c0fabf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:55.003904 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-32d77b74-d02f-4772-ab97-f9ed4c0fabf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:55.013382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d77b74-d02f-4772-ab97-f9ed4c0fabf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:55.014801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d77b74-d02f-4772-ab97-f9ed4c0fabf6 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:55.022974 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1164/2288] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:13:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:13:55.334592 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebbaf6b-951c-4089-81c4-8561ece3b044 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:55.336084 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6ebbaf6b-951c-4089-81c4-8561ece3b044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:55.417699 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebbaf6b-951c-4089-81c4-8561ece3b044 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:55.422731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e59b5-029f-4564-a525-9df7e7b73195 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a0f0fdbbd20492d8c35ec4b73404632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:55.423413 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e59b5-029f-4564-a525-9df7e7b73195 service placement] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:55.424238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e59b5-029f-4564-a525-9df7e7b73195 service placement] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:55.424507 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446e59b5-029f-4564-a525-9df7e7b73195 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:55.426245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-446e59b5-029f-4564-a525-9df7e7b73195 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:55.438368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446e59b5-029f-4564-a525-9df7e7b73195 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:55.440084 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446e59b5-029f-4564-a525-9df7e7b73195 service placement] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:55.450070 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1125/2289] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:13:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:13:57.100947 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d77b74-d02f-4772-ab97-f9ed4c0fabf6 service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:57.102433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-32d77b74-d02f-4772-ab97-f9ed4c0fabf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:57.118585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d77b74-d02f-4772-ab97-f9ed4c0fabf6 service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:57.123988 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5906a79-3710-4181-958a-c9c9d21311f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:57.124574 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5906a79-3710-4181-958a-c9c9d21311f6 service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:57.124848 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5906a79-3710-4181-958a-c9c9d21311f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:57.125118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5906a79-3710-4181-958a-c9c9d21311f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:57.126917 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-d5906a79-3710-4181-958a-c9c9d21311f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:57.136711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5906a79-3710-4181-958a-c9c9d21311f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:57.138262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5906a79-3710-4181-958a-c9c9d21311f6 service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:57.147719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1165/2290] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:13:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:14:05.445720 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=41 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:05.963689 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e59b5-029f-4564-a525-9df7e7b73195 service placement] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:05.972705 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446e59b5-029f-4564-a525-9df7e7b73195 service placement] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:05.976229 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892482be-943c-4967-a140-7dabcc12b436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:05.976820 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892482be-943c-4967-a140-7dabcc12b436 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:05.977100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892482be-943c-4967-a140-7dabcc12b436 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:05.977373 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892482be-943c-4967-a140-7dabcc12b436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:05.979130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-892482be-943c-4967-a140-7dabcc12b436 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:05.988214 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892482be-943c-4967-a140-7dabcc12b436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:05.989617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892482be-943c-4967-a140-7dabcc12b436 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:05.997256 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1126/2291] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:14:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:14:07.144223 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=19 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:08.560353 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5906a79-3710-4181-958a-c9c9d21311f6 service nova] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:08.562295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-d5906a79-3710-4181-958a-c9c9d21311f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:08.575839 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5906a79-3710-4181-958a-c9c9d21311f6 service nova] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:08.581081 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca29e783-4071-4770-8f5d-794bd8c7968a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:08.581750 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29e783-4071-4770-8f5d-794bd8c7968a service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:08.582022 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29e783-4071-4770-8f5d-794bd8c7968a service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:08.582446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca29e783-4071-4770-8f5d-794bd8c7968a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:08.584599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-ca29e783-4071-4770-8f5d-794bd8c7968a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:08.597583 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca29e783-4071-4770-8f5d-794bd8c7968a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:08.599331 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca29e783-4071-4770-8f5d-794bd8c7968a service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:08.609961 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1166/2292] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:14:08 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:14:09.735396 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892482be-943c-4967-a140-7dabcc12b436 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:09.748744 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892482be-943c-4967-a140-7dabcc12b436 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:09.754643 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d363085-a731-42ec-bf45-3ef105d31743 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:09.755307 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d363085-a731-42ec-bf45-3ef105d31743 service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:09.755588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d363085-a731-42ec-bf45-3ef105d31743 service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:09.755860 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d363085-a731-42ec-bf45-3ef105d31743 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:09.758249 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7d363085-a731-42ec-bf45-3ef105d31743 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:09.771745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d363085-a731-42ec-bf45-3ef105d31743 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:09.773344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d363085-a731-42ec-bf45-3ef105d31743 service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:09.784513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1127/2293] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:14:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:14:12.605881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca29e783-4071-4770-8f5d-794bd8c7968a service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:12.623001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca29e783-4071-4770-8f5d-794bd8c7968a service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:12.630133 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf5f16-11d8-46b8-b127-c788472a3b81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:12.630784 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf5f16-11d8-46b8-b127-c788472a3b81 service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:12.631043 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf5f16-11d8-46b8-b127-c788472a3b81 service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:12.631374 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf5f16-11d8-46b8-b127-c788472a3b81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:12.633200 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-cfbf5f16-11d8-46b8-b127-c788472a3b81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:12.647662 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf5f16-11d8-46b8-b127-c788472a3b81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:12.649407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf5f16-11d8-46b8-b127-c788472a3b81 service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:12.662386 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1167/2294] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:14:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:14:19.418753 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f4cdb-aaef-458a-a0a5-df4c5c102f9c None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:19.419310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f4cdb-aaef-458a-a0a5-df4c5c102f9c None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:19.419310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f4cdb-aaef-458a-a0a5-df4c5c102f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:19.438196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-cf4f4cdb-aaef-458a-a0a5-df4c5c102f9c None None] MFA Rules not processed for user `e8b5ce2249a84c01ab57c03b656b2949`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:19.440154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-cf4f4cdb-aaef-458a-a0a5-df4c5c102f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:19.446851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1128/2295] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:14:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:14:22.656579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=12 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:29.432738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=15 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:29.869236 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf5f16-11d8-46b8-b127-c788472a3b81 service nova] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:29.882277 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf5f16-11d8-46b8-b127-c788472a3b81 service nova] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:29.888070 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937abaa-09d1-4a03-ae81-4df0db7ceaac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:29.888682 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937abaa-09d1-4a03-ae81-4df0db7ceaac service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:29.888981 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937abaa-09d1-4a03-ae81-4df0db7ceaac service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:29.889257 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937abaa-09d1-4a03-ae81-4df0db7ceaac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:29.890998 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-1937abaa-09d1-4a03-ae81-4df0db7ceaac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:29.904022 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937abaa-09d1-4a03-ae81-4df0db7ceaac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:29.905671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937abaa-09d1-4a03-ae81-4df0db7ceaac service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:29.918912 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1168/2296] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:14:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:14:32.774312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f4cdb-aaef-458a-a0a5-df4c5c102f9c None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:32.775726 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-cf4f4cdb-aaef-458a-a0a5-df4c5c102f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:32.788042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f4cdb-aaef-458a-a0a5-df4c5c102f9c None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:32.791594 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75a208-a11c-4951-bc47-6f7674ec6f61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:32.792199 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75a208-a11c-4951-bc47-6f7674ec6f61 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:32.792475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75a208-a11c-4951-bc47-6f7674ec6f61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:32.792744 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75a208-a11c-4951-bc47-6f7674ec6f61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:32.794393 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7c75a208-a11c-4951-bc47-6f7674ec6f61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:32.866330 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75a208-a11c-4951-bc47-6f7674ec6f61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:32.867819 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c75a208-a11c-4951-bc47-6f7674ec6f61 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:32.878153 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1129/2297] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:14:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:14:35.770056 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b7262-165b-4985-a6fc-1f252316f374 None None] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:35.770650 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b7262-165b-4985-a6fc-1f252316f374 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:35.770650 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b7262-165b-4985-a6fc-1f252316f374 None None] PATH_INFO: `/` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:35.772010 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1169/2298] 10.222.0.36 () {54 vars in 921 bytes} [Fri Sep 2 09:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:35.778159 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91c006-451f-42d4-9129-36f2333fac39 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:35.778617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91c006-451f-42d4-9129-36f2333fac39 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:35.778700 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91c006-451f-42d4-9129-36f2333fac39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:35.813904 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-7f91c006-451f-42d4-9129-36f2333fac39 None None] MFA Rules not processed for user `07a825fddbc04f918bcd631976038cf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:35.816349 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7f91c006-451f-42d4-9129-36f2333fac39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:35.826370 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1130/2299] 10.222.0.36 () {58 vars in 1018 bytes} [Fri Sep 2 09:14:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:14:35.839246 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b7262-165b-4985-a6fc-1f252316f374 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:35.851691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b7262-165b-4985-a6fc-1f252316f374 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:35.856359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4cabf1-545a-4d76-a6a9-69f5cb3e3e75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:35.857243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cabf1-545a-4d76-a6a9-69f5cb3e3e75 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:35.857687 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cabf1-545a-4d76-a6a9-69f5cb3e3e75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:35.858165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4cabf1-545a-4d76-a6a9-69f5cb3e3e75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:35.872086 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4cabf1-545a-4d76-a6a9-69f5cb3e3e75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:35.873906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4cabf1-545a-4d76-a6a9-69f5cb3e3e75 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:35.883558 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1170/2300] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:14:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:14:35.975849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36f120-8421-4655-b08a-8b85c7c6020f None None] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:35.976209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36f120-8421-4655-b08a-8b85c7c6020f None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:35.976547 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36f120-8421-4655-b08a-8b85c7c6020f None None] PATH_INFO: `/` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:35.978014 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1131/2301] 10.222.0.36 () {54 vars in 921 bytes} [Fri Sep 2 09:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:35.982995 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35184eb5-1245-437a-a37a-99a798c800cd None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:35.983480 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35184eb5-1245-437a-a37a-99a798c800cd None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:35.983765 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35184eb5-1245-437a-a37a-99a798c800cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:36.004306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-35184eb5-1245-437a-a37a-99a798c800cd None None] MFA Rules not processed for user `cce502a83f5343558c2136c0193e7d85`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:36.005504 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-35184eb5-1245-437a-a37a-99a798c800cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:36.014776 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1171/2302] 10.222.0.36 () {58 vars in 1018 bytes} [Fri Sep 2 09:14:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:14:36.034906 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36f120-8421-4655-b08a-8b85c7c6020f None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:36.047881 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36f120-8421-4655-b08a-8b85c7c6020f None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:36.051720 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141e92e-160f-485a-b34e-cf551f58adbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:36.052333 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141e92e-160f-485a-b34e-cf551f58adbe service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:36.052762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141e92e-160f-485a-b34e-cf551f58adbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:36.053205 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141e92e-160f-485a-b34e-cf551f58adbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:36.065652 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7141e92e-160f-485a-b34e-cf551f58adbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:36.066831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7141e92e-160f-485a-b34e-cf551f58adbe service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:36.079501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1132/2303] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:14:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:14:36.659957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35184eb5-1245-437a-a37a-99a798c800cd None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:36.661533 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-35184eb5-1245-437a-a37a-99a798c800cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:36.676979 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35184eb5-1245-437a-a37a-99a798c800cd None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:36.682335 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa78018-ed49-4919-b50c-58f1cc33a89d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a0f0fdbbd20492d8c35ec4b73404632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:36.683020 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa78018-ed49-4919-b50c-58f1cc33a89d service placement] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:36.683405 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa78018-ed49-4919-b50c-58f1cc33a89d service placement] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:36.683747 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa78018-ed49-4919-b50c-58f1cc33a89d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:36.686142 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6fa78018-ed49-4919-b50c-58f1cc33a89d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:36.698997 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa78018-ed49-4919-b50c-58f1cc33a89d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:36.700687 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa78018-ed49-4919-b50c-58f1cc33a89d service placement] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:36.713215 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1172/2304] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:14:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:14:36.723641 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141e92e-160f-485a-b34e-cf551f58adbe service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:36.725447 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7141e92e-160f-485a-b34e-cf551f58adbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:36.749865 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141e92e-160f-485a-b34e-cf551f58adbe service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:36.756637 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339a6a0-c8bb-472a-910d-2bfb1f7f2a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:36.757172 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339a6a0-c8bb-472a-910d-2bfb1f7f2a77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:36.757436 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339a6a0-c8bb-472a-910d-2bfb1f7f2a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:36.757871 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339a6a0-c8bb-472a-910d-2bfb1f7f2a77 admin admin] PATH_INFO: `/v3/users/bf58157cbc8e49c5bd913c68ff859bfb` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:36.759292 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7339a6a0-c8bb-472a-910d-2bfb1f7f2a77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf58157cbc8e49c5bd913c68ff859bfb)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:36.761218 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7339a6a0-c8bb-472a-910d-2bfb1f7f2a77 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:36.810247 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7339a6a0-c8bb-472a-910d-2bfb1f7f2a77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf58157cbc8e49c5bd913c68ff859bfb'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:36.810929 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7339a6a0-c8bb-472a-910d-2bfb1f7f2a77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf58157cbc8e49c5bd913c68ff859bfb'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:36.825831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7339a6a0-c8bb-472a-910d-2bfb1f7f2a77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf58157cbc8e49c5bd913c68ff859bfb'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:36.834915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7339a6a0-c8bb-472a-910d-2bfb1f7f2a77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf58157cbc8e49c5bd913c68ff859bfb'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:36.835734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7339a6a0-c8bb-472a-910d-2bfb1f7f2a77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf58157cbc8e49c5bd913c68ff859bfb'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:36.855173 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1133/2305] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:14:36 2022] DELETE /identity/v3/users/bf58157cbc8e49c5bd913c68ff859bfb => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:37.082473 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa78018-ed49-4919-b50c-58f1cc33a89d service placement] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:37.093310 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa78018-ed49-4919-b50c-58f1cc33a89d service placement] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:37.162302 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bb2cf-1e19-4196-846f-f762b46bf718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:37.163157 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bb2cf-1e19-4196-846f-f762b46bf718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:37.163444 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bb2cf-1e19-4196-846f-f762b46bf718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:37.163722 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bb2cf-1e19-4196-846f-f762b46bf718 admin admin] PATH_INFO: `/v3/projects/6fde620889f34b04bb7c9fcd5ad0a368` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:37.164875 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36bb2cf-1e19-4196-846f-f762b46bf718 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fde620889f34b04bb7c9fcd5ad0a368)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:37.166810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36bb2cf-1e19-4196-846f-f762b46bf718 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:37.216802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e36bb2cf-1e19-4196-846f-f762b46bf718 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fde620889f34b04bb7c9fcd5ad0a368', 'request_id': 'req-e36bb2cf-1e19-4196-846f-f762b46bf718'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:37.224276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e36bb2cf-1e19-4196-846f-f762b46bf718 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fde620889f34b04bb7c9fcd5ad0a368', 'request_id': 'req-e36bb2cf-1e19-4196-846f-f762b46bf718'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:37.229955 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e36bb2cf-1e19-4196-846f-f762b46bf718 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fde620889f34b04bb7c9fcd5ad0a368', 'request_id': 'req-e36bb2cf-1e19-4196-846f-f762b46bf718'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:37.241895 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e36bb2cf-1e19-4196-846f-f762b46bf718 admin admin] The token cache is being invalidate because project 6fde620889f34b04bb7c9fcd5ad0a368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:14:37.242184 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e36bb2cf-1e19-4196-846f-f762b46bf718 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:37.242824 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e36bb2cf-1e19-4196-846f-f762b46bf718 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:37.244324 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1173/2306] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:14:37 2022] DELETE /identity/v3/projects/6fde620889f34b04bb7c9fcd5ad0a368 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:37.250016 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137df60e-bc5e-487d-89b4-91e9ec346610 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:37.250305 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137df60e-bc5e-487d-89b4-91e9ec346610 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:37.250602 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137df60e-bc5e-487d-89b4-91e9ec346610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:37.270471 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-137df60e-bc5e-487d-89b4-91e9ec346610 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:37.272018 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-137df60e-bc5e-487d-89b4-91e9ec346610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:37.316995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:37.337237 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1134/2307] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:14:37.346530 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00ad4b-1893-4ef4-a418-ddf98bc68af5 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:37.346873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00ad4b-1893-4ef4-a418-ddf98bc68af5 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:37.347187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00ad4b-1893-4ef4-a418-ddf98bc68af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:37.368561 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-5e00ad4b-1893-4ef4-a418-ddf98bc68af5 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:37.369786 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5e00ad4b-1893-4ef4-a418-ddf98bc68af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:37.378603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1174/2308] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:14:37.383339 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137df60e-bc5e-487d-89b4-91e9ec346610 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:37.393176 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137df60e-bc5e-487d-89b4-91e9ec346610 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:37.398599 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560cfbd-ec7c-4c6a-a4d5-4b8629756f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:37.399262 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560cfbd-ec7c-4c6a-a4d5-4b8629756f9c admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:37.399574 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560cfbd-ec7c-4c6a-a4d5-4b8629756f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:37.399866 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560cfbd-ec7c-4c6a-a4d5-4b8629756f9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:37.400604 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560cfbd-ec7c-4c6a-a4d5-4b8629756f9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:37.400991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560cfbd-ec7c-4c6a-a4d5-4b8629756f9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:37.402823 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560cfbd-ec7c-4c6a-a4d5-4b8629756f9c admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:37.417053 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1135/2309] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:37.421383 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00ad4b-1893-4ef4-a418-ddf98bc68af5 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:37.431625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00ad4b-1893-4ef4-a418-ddf98bc68af5 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:37.436730 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5ea42-8565-4d44-bc37-0fbcbd72e851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:37.437367 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5ea42-8565-4d44-bc37-0fbcbd72e851 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:37.437677 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5ea42-8565-4d44-bc37-0fbcbd72e851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:37.438037 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5ea42-8565-4d44-bc37-0fbcbd72e851 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:37.438867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5ea42-8565-4d44-bc37-0fbcbd72e851 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:37.440877 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a5ea42-8565-4d44-bc37-0fbcbd72e851 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:37.470661 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1175/2310] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:14:37 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:14:37.475896 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560cfbd-ec7c-4c6a-a4d5-4b8629756f9c admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:37.486079 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560cfbd-ec7c-4c6a-a4d5-4b8629756f9c admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:37.545966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248d67c-035b-4dd2-af71-dbb00006fd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:37.546511 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248d67c-035b-4dd2-af71-dbb00006fd5b admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:37.546795 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248d67c-035b-4dd2-af71-dbb00006fd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:37.547091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248d67c-035b-4dd2-af71-dbb00006fd5b admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:37.547750 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248d67c-035b-4dd2-af71-dbb00006fd5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:37.550488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248d67c-035b-4dd2-af71-dbb00006fd5b admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:37.576488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1136/2311] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:14:37 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:14:37.577655 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5ea42-8565-4d44-bc37-0fbcbd72e851 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:37.588242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5ea42-8565-4d44-bc37-0fbcbd72e851 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:37.593556 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1628e1f-05d7-4c0c-98e7-bdaf8e5f25d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:37.594276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1628e1f-05d7-4c0c-98e7-bdaf8e5f25d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:37.594555 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1628e1f-05d7-4c0c-98e7-bdaf8e5f25d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:37.594856 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1628e1f-05d7-4c0c-98e7-bdaf8e5f25d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:37.595534 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1628e1f-05d7-4c0c-98e7-bdaf8e5f25d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:37.597835 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1628e1f-05d7-4c0c-98e7-bdaf8e5f25d5 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:37.612485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1176/2312] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:14:37.613650 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248d67c-035b-4dd2-af71-dbb00006fd5b admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:37.623980 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248d67c-035b-4dd2-af71-dbb00006fd5b admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:37.628562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11dca0-45e2-4752-b17b-a1b189b2e36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:37.629237 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11dca0-45e2-4752-b17b-a1b189b2e36c admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:37.629470 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11dca0-45e2-4752-b17b-a1b189b2e36c admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:37.629716 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11dca0-45e2-4752-b17b-a1b189b2e36c admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:37.630352 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b11dca0-45e2-4752-b17b-a1b189b2e36c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:37.631970 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b11dca0-45e2-4752-b17b-a1b189b2e36c admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:37.644256 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1137/2313] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:14:37.647464 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1628e1f-05d7-4c0c-98e7-bdaf8e5f25d5 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:37.653769 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:37.657945 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1628e1f-05d7-4c0c-98e7-bdaf8e5f25d5 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:37.664562 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d9c27-0947-4209-a2ba-bc9c40758315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:37.665268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d9c27-0947-4209-a2ba-bc9c40758315 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:37.665756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d9c27-0947-4209-a2ba-bc9c40758315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:37.666202 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d9c27-0947-4209-a2ba-bc9c40758315 admin admin] PATH_INFO: `/v3/projects/0c1196cd224a4bd4828eaeb87e6a34e7/users/e2fbcbc4f55c4545afa2c5b08700738b/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:37.681645 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d9c27-0947-4209-a2ba-bc9c40758315 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c1196cd224a4bd4828eaeb87e6a34e7, user_id=e2fbcbc4f55c4545afa2c5b08700738b, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:37.684347 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d9c27-0947-4209-a2ba-bc9c40758315 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:37.697887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1177/2314] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:14:37 2022] PUT /identity/v3/projects/0c1196cd224a4bd4828eaeb87e6a34e7/users/e2fbcbc4f55c4545afa2c5b08700738b/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:37.709457 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9b521-e554-4a1d-993e-bf4b800d59e5 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:37.709749 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9b521-e554-4a1d-993e-bf4b800d59e5 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:37.710025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9b521-e554-4a1d-993e-bf4b800d59e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:37.737895 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-3be9b521-e554-4a1d-993e-bf4b800d59e5 None None] MFA Rules not processed for user `e2fbcbc4f55c4545afa2c5b08700738b`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:37.739127 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-3be9b521-e554-4a1d-993e-bf4b800d59e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:37.804515 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1138/2315] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:14:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:14:37.809273 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d9c27-0947-4209-a2ba-bc9c40758315 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:37.876505 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d9c27-0947-4209-a2ba-bc9c40758315 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:37.881276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59752853-bf53-46a2-87ae-844aeacaed1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:37.881879 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59752853-bf53-46a2-87ae-844aeacaed1d admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:37.882155 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59752853-bf53-46a2-87ae-844aeacaed1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:37.882426 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59752853-bf53-46a2-87ae-844aeacaed1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:37.883098 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59752853-bf53-46a2-87ae-844aeacaed1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:37.884997 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59752853-bf53-46a2-87ae-844aeacaed1d admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:37.914250 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1178/2316] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:14:37 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:14:37.918551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9b521-e554-4a1d-993e-bf4b800d59e5 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:37.930233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9b521-e554-4a1d-993e-bf4b800d59e5 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:37.969924 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:37.987810 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e9283-0fcd-4c43-91e2-4da80646f07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:37.988407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e9283-0fcd-4c43-91e2-4da80646f07c admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:37.988684 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e9283-0fcd-4c43-91e2-4da80646f07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:37.988951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e9283-0fcd-4c43-91e2-4da80646f07c admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:37.989571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e9283-0fcd-4c43-91e2-4da80646f07c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:37.991498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e9283-0fcd-4c43-91e2-4da80646f07c admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:38.016568 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1139/2317] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:14:37 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:14:38.018417 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59752853-bf53-46a2-87ae-844aeacaed1d admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:38.028790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59752853-bf53-46a2-87ae-844aeacaed1d admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:38.033524 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc6a7c-1510-428d-832c-ddff64e580b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:38.034115 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc6a7c-1510-428d-832c-ddff64e580b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:38.034397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc6a7c-1510-428d-832c-ddff64e580b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:38.034672 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc6a7c-1510-428d-832c-ddff64e580b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:38.035342 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc6a7c-1510-428d-832c-ddff64e580b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:38.036941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc6a7c-1510-428d-832c-ddff64e580b1 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:38.047317 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1179/2318] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:14:38.051815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e9283-0fcd-4c43-91e2-4da80646f07c admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:38.061576 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e9283-0fcd-4c43-91e2-4da80646f07c admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:38.066621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2da5ae-7fb3-453e-9f7d-cd9238313e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:38.067290 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2da5ae-7fb3-453e-9f7d-cd9238313e4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:38.067573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2da5ae-7fb3-453e-9f7d-cd9238313e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:38.067840 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2da5ae-7fb3-453e-9f7d-cd9238313e4b admin admin] PATH_INFO: `/v3/projects/938b9389a0774524a0e49947edd9547e/users/b90028dd5cbb4ebea30e1a6943df4a75/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:38.084169 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2da5ae-7fb3-453e-9f7d-cd9238313e4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=938b9389a0774524a0e49947edd9547e, user_id=b90028dd5cbb4ebea30e1a6943df4a75, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:38.086814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2da5ae-7fb3-453e-9f7d-cd9238313e4b admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:38.100469 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1140/2319] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:14:38 2022] PUT /identity/v3/projects/938b9389a0774524a0e49947edd9547e/users/b90028dd5cbb4ebea30e1a6943df4a75/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:14:38.104301 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc6a7c-1510-428d-832c-ddff64e580b1 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:38.114310 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc6a7c-1510-428d-832c-ddff64e580b1 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:38.174147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2a811-4405-4fa4-b080-e8f4da4e8ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:38.174792 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2a811-4405-4fa4-b080-e8f4da4e8ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:38.175098 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2a811-4405-4fa4-b080-e8f4da4e8ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:38.175378 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2a811-4405-4fa4-b080-e8f4da4e8ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:38.176022 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2a811-4405-4fa4-b080-e8f4da4e8ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:38.177627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2a811-4405-4fa4-b080-e8f4da4e8ac4 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:38.188092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1180/2320] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:14:38.192597 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2da5ae-7fb3-453e-9f7d-cd9238313e4b admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:38.203207 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2da5ae-7fb3-453e-9f7d-cd9238313e4b admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:38.208983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f22be-bfdb-4cb0-bab7-99a01782e8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:38.209661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f22be-bfdb-4cb0-bab7-99a01782e8d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:38.209944 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f22be-bfdb-4cb0-bab7-99a01782e8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:38.210218 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277f22be-bfdb-4cb0-bab7-99a01782e8d5 admin admin] PATH_INFO: `/v3/projects/938b9389a0774524a0e49947edd9547e/users/b90028dd5cbb4ebea30e1a6943df4a75/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:38.212224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f22be-bfdb-4cb0-bab7-99a01782e8d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=938b9389a0774524a0e49947edd9547e, user_id=b90028dd5cbb4ebea30e1a6943df4a75, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:38.214893 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277f22be-bfdb-4cb0-bab7-99a01782e8d5 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:38.225991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1141/2321] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:14:38 2022] PUT /identity/v3/projects/938b9389a0774524a0e49947edd9547e/users/b90028dd5cbb4ebea30e1a6943df4a75/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:38.236107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a199754-17c8-410a-bab2-268ca793802e None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:38.236704 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a199754-17c8-410a-bab2-268ca793802e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:38.236863 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a199754-17c8-410a-bab2-268ca793802e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:38.265336 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-5a199754-17c8-410a-bab2-268ca793802e None None] MFA Rules not processed for user `b90028dd5cbb4ebea30e1a6943df4a75`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:38.266588 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5a199754-17c8-410a-bab2-268ca793802e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:38.308873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:38.340991 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1181/2322] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:14:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:14:38.353507 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f22be-bfdb-4cb0-bab7-99a01782e8d5 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:38.355414 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-277f22be-bfdb-4cb0-bab7-99a01782e8d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:38.415309 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277f22be-bfdb-4cb0-bab7-99a01782e8d5 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:38.418981 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eba4c1-36aa-41e5-ae07-74a2f52ad617 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:38.419623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eba4c1-36aa-41e5-ae07-74a2f52ad617 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:38.419906 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eba4c1-36aa-41e5-ae07-74a2f52ad617 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:38.420175 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eba4c1-36aa-41e5-ae07-74a2f52ad617 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:38.429974 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eba4c1-36aa-41e5-ae07-74a2f52ad617 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:38.431455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eba4c1-36aa-41e5-ae07-74a2f52ad617 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:38.441792 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1142/2323] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:14:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:14:38.833498 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a199754-17c8-410a-bab2-268ca793802e None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:38.894277 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a199754-17c8-410a-bab2-268ca793802e None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:38.898685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2421e01-5914-47c3-b5b6-752f1e1f1a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:38.899326 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2421e01-5914-47c3-b5b6-752f1e1f1a88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:38.899610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2421e01-5914-47c3-b5b6-752f1e1f1a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:38.899873 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2421e01-5914-47c3-b5b6-752f1e1f1a88 admin admin] PATH_INFO: `/v3/users/e2fbcbc4f55c4545afa2c5b08700738b` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:38.900955 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2421e01-5914-47c3-b5b6-752f1e1f1a88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2fbcbc4f55c4545afa2c5b08700738b)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:38.902893 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2421e01-5914-47c3-b5b6-752f1e1f1a88 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:38.947920 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-a2421e01-5914-47c3-b5b6-752f1e1f1a88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2fbcbc4f55c4545afa2c5b08700738b'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:38.955304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-a2421e01-5914-47c3-b5b6-752f1e1f1a88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2fbcbc4f55c4545afa2c5b08700738b'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:38.962650 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-a2421e01-5914-47c3-b5b6-752f1e1f1a88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2fbcbc4f55c4545afa2c5b08700738b'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:38.963310 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-a2421e01-5914-47c3-b5b6-752f1e1f1a88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2fbcbc4f55c4545afa2c5b08700738b'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:38.964013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-a2421e01-5914-47c3-b5b6-752f1e1f1a88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2fbcbc4f55c4545afa2c5b08700738b'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:38.980949 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1182/2324] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:14:38 2022] DELETE /identity/v3/users/e2fbcbc4f55c4545afa2c5b08700738b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:38.985359 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7f05b-f8c3-4e0a-b77b-d3396f7137c2 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:38.985861 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7f05b-f8c3-4e0a-b77b-d3396f7137c2 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:38.985976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7f05b-f8c3-4e0a-b77b-d3396f7137c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:39.004595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-9fd7f05b-f8c3-4e0a-b77b-d3396f7137c2 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:39.005801 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-9fd7f05b-f8c3-4e0a-b77b-d3396f7137c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:39.061870 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1143/2325] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:14:39.071492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2421e01-5914-47c3-b5b6-752f1e1f1a88 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:39.072906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a2421e01-5914-47c3-b5b6-752f1e1f1a88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:39.130783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2421e01-5914-47c3-b5b6-752f1e1f1a88 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:39.134393 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a9431-9831-4a9e-a735-9ab393fef19d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:39.134989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a9431-9831-4a9e-a735-9ab393fef19d service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:39.135297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a9431-9831-4a9e-a735-9ab393fef19d service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:39.135570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a9431-9831-4a9e-a735-9ab393fef19d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:39.145309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a9431-9831-4a9e-a735-9ab393fef19d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:39.146680 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a9431-9831-4a9e-a735-9ab393fef19d service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:39.150652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:39.160821 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1183/2326] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:14:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:14:39.428220 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7f05b-f8c3-4e0a-b77b-d3396f7137c2 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:39.429801 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-9fd7f05b-f8c3-4e0a-b77b-d3396f7137c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:39.447599 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7f05b-f8c3-4e0a-b77b-d3396f7137c2 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:39.452576 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7846816-286f-4c90-af2c-bc1af0aecbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:39.453219 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7846816-286f-4c90-af2c-bc1af0aecbc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:39.453585 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7846816-286f-4c90-af2c-bc1af0aecbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:39.453775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7846816-286f-4c90-af2c-bc1af0aecbc5 admin admin] PATH_INFO: `/v3/projects/0c1196cd224a4bd4828eaeb87e6a34e7` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:39.455200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7846816-286f-4c90-af2c-bc1af0aecbc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c1196cd224a4bd4828eaeb87e6a34e7)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:39.457321 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7846816-286f-4c90-af2c-bc1af0aecbc5 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:39.493907 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:39.507261 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a7846816-286f-4c90-af2c-bc1af0aecbc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c1196cd224a4bd4828eaeb87e6a34e7', 'request_id': 'req-a7846816-286f-4c90-af2c-bc1af0aecbc5'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:39.513315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a7846816-286f-4c90-af2c-bc1af0aecbc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c1196cd224a4bd4828eaeb87e6a34e7', 'request_id': 'req-a7846816-286f-4c90-af2c-bc1af0aecbc5'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:39.519028 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a7846816-286f-4c90-af2c-bc1af0aecbc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c1196cd224a4bd4828eaeb87e6a34e7', 'request_id': 'req-a7846816-286f-4c90-af2c-bc1af0aecbc5'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:39.531854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a7846816-286f-4c90-af2c-bc1af0aecbc5 admin admin] The token cache is being invalidate because project 0c1196cd224a4bd4828eaeb87e6a34e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:14:39.532383 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a7846816-286f-4c90-af2c-bc1af0aecbc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:39.533014 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a7846816-286f-4c90-af2c-bc1af0aecbc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:39.535312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1144/2327] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:14:39 2022] DELETE /identity/v3/projects/0c1196cd224a4bd4828eaeb87e6a34e7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:39.539227 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a9431-9831-4a9e-a735-9ab393fef19d service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:39.549760 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a9431-9831-4a9e-a735-9ab393fef19d service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:39.605935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5cf951-7f54-4fa3-8f5e-4715af359a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:39.606555 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5cf951-7f54-4fa3-8f5e-4715af359a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:39.606829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5cf951-7f54-4fa3-8f5e-4715af359a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:39.607118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5cf951-7f54-4fa3-8f5e-4715af359a8b admin admin] PATH_INFO: `/v3/users/b90028dd5cbb4ebea30e1a6943df4a75` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:39.608214 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5cf951-7f54-4fa3-8f5e-4715af359a8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b90028dd5cbb4ebea30e1a6943df4a75)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:39.610119 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5cf951-7f54-4fa3-8f5e-4715af359a8b admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:39.667947 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-cb5cf951-7f54-4fa3-8f5e-4715af359a8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b90028dd5cbb4ebea30e1a6943df4a75'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:39.675991 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-cb5cf951-7f54-4fa3-8f5e-4715af359a8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b90028dd5cbb4ebea30e1a6943df4a75'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:39.682628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-cb5cf951-7f54-4fa3-8f5e-4715af359a8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b90028dd5cbb4ebea30e1a6943df4a75'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:39.683292 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-cb5cf951-7f54-4fa3-8f5e-4715af359a8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b90028dd5cbb4ebea30e1a6943df4a75'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:39.683887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-cb5cf951-7f54-4fa3-8f5e-4715af359a8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b90028dd5cbb4ebea30e1a6943df4a75'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:39.701094 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1184/2328] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:14:39 2022] DELETE /identity/v3/users/b90028dd5cbb4ebea30e1a6943df4a75 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:39.971557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7846816-286f-4c90-af2c-bc1af0aecbc5 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:39.973088 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-a7846816-286f-4c90-af2c-bc1af0aecbc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:39.986969 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7846816-286f-4c90-af2c-bc1af0aecbc5 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:40.044668 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c44ba1-094d-4c7a-a76d-b3a836acbfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:40.045307 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c44ba1-094d-4c7a-a76d-b3a836acbfbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.045586 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c44ba1-094d-4c7a-a76d-b3a836acbfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.045858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c44ba1-094d-4c7a-a76d-b3a836acbfbf admin admin] PATH_INFO: `/v3/projects/938b9389a0774524a0e49947edd9547e` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.046968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c44ba1-094d-4c7a-a76d-b3a836acbfbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=938b9389a0774524a0e49947edd9547e)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:40.048928 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c44ba1-094d-4c7a-a76d-b3a836acbfbf admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:40.094882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-90c44ba1-094d-4c7a-a76d-b3a836acbfbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '938b9389a0774524a0e49947edd9547e', 'request_id': 'req-90c44ba1-094d-4c7a-a76d-b3a836acbfbf'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:40.101535 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-90c44ba1-094d-4c7a-a76d-b3a836acbfbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '938b9389a0774524a0e49947edd9547e', 'request_id': 'req-90c44ba1-094d-4c7a-a76d-b3a836acbfbf'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:40.106971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-90c44ba1-094d-4c7a-a76d-b3a836acbfbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '938b9389a0774524a0e49947edd9547e', 'request_id': 'req-90c44ba1-094d-4c7a-a76d-b3a836acbfbf'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:40.118960 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-90c44ba1-094d-4c7a-a76d-b3a836acbfbf admin admin] The token cache is being invalidate because project 938b9389a0774524a0e49947edd9547e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:14:40.119274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-90c44ba1-094d-4c7a-a76d-b3a836acbfbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:40.119891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-90c44ba1-094d-4c7a-a76d-b3a836acbfbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:40.121379 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1145/2329] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:14:39 2022] DELETE /identity/v3/projects/938b9389a0774524a0e49947edd9547e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:40.126396 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caed309-0646-4656-8961-dbf772ef7e7a None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.126679 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caed309-0646-4656-8961-dbf772ef7e7a None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.126948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caed309-0646-4656-8961-dbf772ef7e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.147547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-6caed309-0646-4656-8961-dbf772ef7e7a None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:40.148716 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6caed309-0646-4656-8961-dbf772ef7e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:40.204631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1185/2330] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:14:40.213251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe17d03-6a09-41a3-a738-19e2fc850a1f None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.213539 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe17d03-6a09-41a3-a738-19e2fc850a1f None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.213815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe17d03-6a09-41a3-a738-19e2fc850a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.233753 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-ffe17d03-6a09-41a3-a738-19e2fc850a1f None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:40.234945 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ffe17d03-6a09-41a3-a738-19e2fc850a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:40.242793 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1146/2331] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:14:40.246883 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caed309-0646-4656-8961-dbf772ef7e7a None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:40.256897 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caed309-0646-4656-8961-dbf772ef7e7a None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:40.261304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9184ac-587e-4877-9c71-f7adc210b7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:40.261882 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9184ac-587e-4877-9c71-f7adc210b7cb admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.262158 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9184ac-587e-4877-9c71-f7adc210b7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.262426 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9184ac-587e-4877-9c71-f7adc210b7cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.263055 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9184ac-587e-4877-9c71-f7adc210b7cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:40.263383 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9184ac-587e-4877-9c71-f7adc210b7cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:40.264957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9184ac-587e-4877-9c71-f7adc210b7cb admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:40.276718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1186/2332] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:40.280803 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe17d03-6a09-41a3-a738-19e2fc850a1f None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:40.289897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe17d03-6a09-41a3-a738-19e2fc850a1f None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:40.294478 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68bd505-0eff-4952-aa13-fc9f6aff1810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:40.295059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bd505-0eff-4952-aa13-fc9f6aff1810 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.295362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bd505-0eff-4952-aa13-fc9f6aff1810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.295629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bd505-0eff-4952-aa13-fc9f6aff1810 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.296243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68bd505-0eff-4952-aa13-fc9f6aff1810 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:40.298109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68bd505-0eff-4952-aa13-fc9f6aff1810 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:40.338629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1147/2333] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:14:40 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:14:40.342992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9184ac-587e-4877-9c71-f7adc210b7cb admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:40.349262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:40.353379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9184ac-587e-4877-9c71-f7adc210b7cb admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:40.411403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccebb3a-40a3-41a9-bf5a-26e79896b5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:40.412006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccebb3a-40a3-41a9-bf5a-26e79896b5d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.412289 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccebb3a-40a3-41a9-bf5a-26e79896b5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.412560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccebb3a-40a3-41a9-bf5a-26e79896b5d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.413186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccebb3a-40a3-41a9-bf5a-26e79896b5d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:40.415157 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccebb3a-40a3-41a9-bf5a-26e79896b5d6 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:40.435903 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1187/2334] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:14:40 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:14:40.440483 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68bd505-0eff-4952-aa13-fc9f6aff1810 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:40.450691 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68bd505-0eff-4952-aa13-fc9f6aff1810 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:40.456297 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef633b0-a56c-4a26-85d1-3cb8ca0b94ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:40.456839 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef633b0-a56c-4a26-85d1-3cb8ca0b94ae admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.457127 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef633b0-a56c-4a26-85d1-3cb8ca0b94ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.457395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef633b0-a56c-4a26-85d1-3cb8ca0b94ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.458041 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef633b0-a56c-4a26-85d1-3cb8ca0b94ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:40.459656 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef633b0-a56c-4a26-85d1-3cb8ca0b94ae admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:40.470119 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1148/2335] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:14:40.474260 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccebb3a-40a3-41a9-bf5a-26e79896b5d6 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:40.484109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccebb3a-40a3-41a9-bf5a-26e79896b5d6 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:40.488718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c928cd-d21d-4120-949e-1bb2270ab89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:40.489329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c928cd-d21d-4120-949e-1bb2270ab89e admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.489603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c928cd-d21d-4120-949e-1bb2270ab89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.489867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c928cd-d21d-4120-949e-1bb2270ab89e admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.490493 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c928cd-d21d-4120-949e-1bb2270ab89e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:40.492101 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c928cd-d21d-4120-949e-1bb2270ab89e admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:40.502838 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1188/2336] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:14:40.506836 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef633b0-a56c-4a26-85d1-3cb8ca0b94ae admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:40.516771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef633b0-a56c-4a26-85d1-3cb8ca0b94ae admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:40.521927 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5a988-118a-420f-8300-7045ac72239f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:40.522567 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5a988-118a-420f-8300-7045ac72239f admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.522851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5a988-118a-420f-8300-7045ac72239f admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.523167 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5a988-118a-420f-8300-7045ac72239f admin admin] PATH_INFO: `/v3/projects/f0389d79f6bd420ba06f8a2b72e380a3/users/eaa0c86a954841a1a314bd1e0fd394fe/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.531296 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:40.539545 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5a988-118a-420f-8300-7045ac72239f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0389d79f6bd420ba06f8a2b72e380a3, user_id=eaa0c86a954841a1a314bd1e0fd394fe, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:40.542474 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5a988-118a-420f-8300-7045ac72239f admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:40.554238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1149/2337] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:14:40 2022] PUT /identity/v3/projects/f0389d79f6bd420ba06f8a2b72e380a3/users/eaa0c86a954841a1a314bd1e0fd394fe/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:40.565934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf66d08-d139-4deb-967c-5157d72dc49c None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.566239 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf66d08-d139-4deb-967c-5157d72dc49c None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.566492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf66d08-d139-4deb-967c-5157d72dc49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.594093 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-9cf66d08-d139-4deb-967c-5157d72dc49c None None] MFA Rules not processed for user `eaa0c86a954841a1a314bd1e0fd394fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:40.595316 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-9cf66d08-d139-4deb-967c-5157d72dc49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:40.659120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1189/2338] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:14:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:14:40.663897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5a988-118a-420f-8300-7045ac72239f admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:40.725617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5a988-118a-420f-8300-7045ac72239f admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:40.730453 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd016f06-3946-4997-8de4-73a05a2b3e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:40.731090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd016f06-3946-4997-8de4-73a05a2b3e21 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.731374 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd016f06-3946-4997-8de4-73a05a2b3e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.731648 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd016f06-3946-4997-8de4-73a05a2b3e21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.732322 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd016f06-3946-4997-8de4-73a05a2b3e21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:40.734225 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd016f06-3946-4997-8de4-73a05a2b3e21 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:40.762630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1150/2339] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:14:40 2022] POST /identity/v3/projects => generated 363 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:14:40.766630 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf66d08-d139-4deb-967c-5157d72dc49c None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:40.777206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf66d08-d139-4deb-967c-5157d72dc49c None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:40.833393 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c3c74-e495-4bda-832b-3af83cf313f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:40.833997 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c3c74-e495-4bda-832b-3af83cf313f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.834279 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c3c74-e495-4bda-832b-3af83cf313f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.834551 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c3c74-e495-4bda-832b-3af83cf313f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.835223 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c3c74-e495-4bda-832b-3af83cf313f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:40.837146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c3c74-e495-4bda-832b-3af83cf313f7 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:40.856894 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1190/2340] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:14:40 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:14:40.860925 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd016f06-3946-4997-8de4-73a05a2b3e21 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:40.871521 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd016f06-3946-4997-8de4-73a05a2b3e21 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:40.876050 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05abb0-aff3-4d97-ab4c-f32ae7609318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:40.876634 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05abb0-aff3-4d97-ab4c-f32ae7609318 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.876933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05abb0-aff3-4d97-ab4c-f32ae7609318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.877844 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05abb0-aff3-4d97-ab4c-f32ae7609318 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.878481 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05abb0-aff3-4d97-ab4c-f32ae7609318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:40.880081 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05abb0-aff3-4d97-ab4c-f32ae7609318 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:40.891197 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1151/2341] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:14:40.895024 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c3c74-e495-4bda-832b-3af83cf313f7 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:40.900190 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:40.906418 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c3c74-e495-4bda-832b-3af83cf313f7 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:40.911465 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9ef72-122e-431d-97c8-3970ce06210b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:40.912113 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9ef72-122e-431d-97c8-3970ce06210b admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.912395 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9ef72-122e-431d-97c8-3970ce06210b admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.912672 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9ef72-122e-431d-97c8-3970ce06210b admin admin] PATH_INFO: `/v3/projects/b74133d544994ab698fe82f7d8da2bd5/users/42f89d219cc34c5b9d24ebcb1cef7ce7/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.927544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9ef72-122e-431d-97c8-3970ce06210b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b74133d544994ab698fe82f7d8da2bd5, user_id=42f89d219cc34c5b9d24ebcb1cef7ce7, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:40.930197 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9ef72-122e-431d-97c8-3970ce06210b admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:40.942839 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1191/2342] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:14:40 2022] PUT /identity/v3/projects/b74133d544994ab698fe82f7d8da2bd5/users/42f89d219cc34c5b9d24ebcb1cef7ce7/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:40.946658 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05abb0-aff3-4d97-ab4c-f32ae7609318 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:40.957017 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05abb0-aff3-4d97-ab4c-f32ae7609318 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:41.011169 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59817dd-5721-4ce1-91f9-34ed984bf25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:41.011767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59817dd-5721-4ce1-91f9-34ed984bf25a admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:41.011949 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59817dd-5721-4ce1-91f9-34ed984bf25a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:41.012218 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59817dd-5721-4ce1-91f9-34ed984bf25a admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:41.012848 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59817dd-5721-4ce1-91f9-34ed984bf25a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:41.014432 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59817dd-5721-4ce1-91f9-34ed984bf25a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:41.023829 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1152/2343] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:14:41.027923 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9ef72-122e-431d-97c8-3970ce06210b admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:41.038273 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9ef72-122e-431d-97c8-3970ce06210b admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:41.042763 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712f4d5-fa84-4839-a106-e0bbc677034a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:41.043445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712f4d5-fa84-4839-a106-e0bbc677034a admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:41.043723 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712f4d5-fa84-4839-a106-e0bbc677034a admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:41.043993 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712f4d5-fa84-4839-a106-e0bbc677034a admin admin] PATH_INFO: `/v3/projects/b74133d544994ab698fe82f7d8da2bd5/users/42f89d219cc34c5b9d24ebcb1cef7ce7/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:41.045891 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8712f4d5-fa84-4839-a106-e0bbc677034a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b74133d544994ab698fe82f7d8da2bd5, user_id=42f89d219cc34c5b9d24ebcb1cef7ce7, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:41.048549 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8712f4d5-fa84-4839-a106-e0bbc677034a admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:41.058999 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1192/2344] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:14:41 2022] PUT /identity/v3/projects/b74133d544994ab698fe82f7d8da2bd5/users/42f89d219cc34c5b9d24ebcb1cef7ce7/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:41.068915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05461fda-6f45-4b8b-b781-2e970625890f None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:41.069206 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05461fda-6f45-4b8b-b781-2e970625890f None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:41.069530 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05461fda-6f45-4b8b-b781-2e970625890f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:41.097358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-05461fda-6f45-4b8b-b781-2e970625890f None None] MFA Rules not processed for user `42f89d219cc34c5b9d24ebcb1cef7ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:41.098573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-05461fda-6f45-4b8b-b781-2e970625890f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:41.141645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:41.165327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1153/2345] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:14:41 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:14:41.177016 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712f4d5-fa84-4839-a106-e0bbc677034a admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:41.178762 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8712f4d5-fa84-4839-a106-e0bbc677034a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:41.234635 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712f4d5-fa84-4839-a106-e0bbc677034a admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:41.238894 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a289ac8-2732-4c61-b6e0-607a20a4b0cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:41.239514 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a289ac8-2732-4c61-b6e0-607a20a4b0cd service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:41.239790 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a289ac8-2732-4c61-b6e0-607a20a4b0cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:41.240057 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a289ac8-2732-4c61-b6e0-607a20a4b0cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:41.248644 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a289ac8-2732-4c61-b6e0-607a20a4b0cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:41.250061 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a289ac8-2732-4c61-b6e0-607a20a4b0cd service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:41.259214 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1193/2346] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:14:41 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:14:41.334113 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05461fda-6f45-4b8b-b781-2e970625890f None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:41.343767 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05461fda-6f45-4b8b-b781-2e970625890f None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:41.347926 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b2753-74d5-46b9-851f-ad41e9da883a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:41.348555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b2753-74d5-46b9-851f-ad41e9da883a service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:41.348835 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b2753-74d5-46b9-851f-ad41e9da883a service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:41.349108 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b2753-74d5-46b9-851f-ad41e9da883a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:41.404765 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b2753-74d5-46b9-851f-ad41e9da883a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:41.406182 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b2753-74d5-46b9-851f-ad41e9da883a service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:41.415559 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1154/2347] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:14:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:14:44.145744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a289ac8-2732-4c61-b6e0-607a20a4b0cd service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:44.218775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a289ac8-2732-4c61-b6e0-607a20a4b0cd service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:44.223343 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea6f06-a06c-4a16-9745-5b8b80a8af11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:44.223968 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea6f06-a06c-4a16-9745-5b8b80a8af11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:44.224242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea6f06-a06c-4a16-9745-5b8b80a8af11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:44.224517 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea6f06-a06c-4a16-9745-5b8b80a8af11 admin admin] PATH_INFO: `/v3/users/eaa0c86a954841a1a314bd1e0fd394fe` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:44.225619 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea6f06-a06c-4a16-9745-5b8b80a8af11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaa0c86a954841a1a314bd1e0fd394fe)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:44.227580 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea6f06-a06c-4a16-9745-5b8b80a8af11 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:44.275650 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-56ea6f06-a06c-4a16-9745-5b8b80a8af11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaa0c86a954841a1a314bd1e0fd394fe'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:44.282446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-56ea6f06-a06c-4a16-9745-5b8b80a8af11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaa0c86a954841a1a314bd1e0fd394fe'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:44.289450 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-56ea6f06-a06c-4a16-9745-5b8b80a8af11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaa0c86a954841a1a314bd1e0fd394fe'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:44.290057 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-56ea6f06-a06c-4a16-9745-5b8b80a8af11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaa0c86a954841a1a314bd1e0fd394fe'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:44.290650 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-56ea6f06-a06c-4a16-9745-5b8b80a8af11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaa0c86a954841a1a314bd1e0fd394fe'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:44.295827 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:44.310448 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1194/2348] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:14:44 2022] DELETE /identity/v3/users/eaa0c86a954841a1a314bd1e0fd394fe => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:44.315331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e65f0-d6e4-4696-b6b0-cd141d6667af None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:44.315820 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e65f0-d6e4-4696-b6b0-cd141d6667af None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:44.315999 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e65f0-d6e4-4696-b6b0-cd141d6667af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:44.335778 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-1b6e65f0-d6e4-4696-b6b0-cd141d6667af None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:44.336975 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1b6e65f0-d6e4-4696-b6b0-cd141d6667af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:44.397710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1155/2349] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:14:44.407356 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea6f06-a06c-4a16-9745-5b8b80a8af11 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:44.408735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-56ea6f06-a06c-4a16-9745-5b8b80a8af11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:44.468190 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea6f06-a06c-4a16-9745-5b8b80a8af11 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:44.471810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275eedf-745a-4421-9f94-0fa02dbd6377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:44.472409 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275eedf-745a-4421-9f94-0fa02dbd6377 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:44.472687 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275eedf-745a-4421-9f94-0fa02dbd6377 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:44.472953 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275eedf-745a-4421-9f94-0fa02dbd6377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:44.481728 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275eedf-745a-4421-9f94-0fa02dbd6377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:44.483172 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275eedf-745a-4421-9f94-0fa02dbd6377 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:44.492778 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1195/2350] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:14:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:14:44.752718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e65f0-d6e4-4696-b6b0-cd141d6667af None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:44.754150 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1b6e65f0-d6e4-4696-b6b0-cd141d6667af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:44.768082 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e65f0-d6e4-4696-b6b0-cd141d6667af None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:44.772715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23019c-124b-4419-8fd9-e129d7b25b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:44.773345 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23019c-124b-4419-8fd9-e129d7b25b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:44.773624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23019c-124b-4419-8fd9-e129d7b25b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:44.773897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23019c-124b-4419-8fd9-e129d7b25b6f admin admin] PATH_INFO: `/v3/projects/f0389d79f6bd420ba06f8a2b72e380a3` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:44.775016 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23019c-124b-4419-8fd9-e129d7b25b6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0389d79f6bd420ba06f8a2b72e380a3)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:44.776951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23019c-124b-4419-8fd9-e129d7b25b6f admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:44.824705 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-6b23019c-124b-4419-8fd9-e129d7b25b6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0389d79f6bd420ba06f8a2b72e380a3', 'request_id': 'req-6b23019c-124b-4419-8fd9-e129d7b25b6f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:44.832332 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-6b23019c-124b-4419-8fd9-e129d7b25b6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0389d79f6bd420ba06f8a2b72e380a3', 'request_id': 'req-6b23019c-124b-4419-8fd9-e129d7b25b6f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:44.837700 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-6b23019c-124b-4419-8fd9-e129d7b25b6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0389d79f6bd420ba06f8a2b72e380a3', 'request_id': 'req-6b23019c-124b-4419-8fd9-e129d7b25b6f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:44.850230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-6b23019c-124b-4419-8fd9-e129d7b25b6f admin admin] The token cache is being invalidate because project f0389d79f6bd420ba06f8a2b72e380a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:14:44.850971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-6b23019c-124b-4419-8fd9-e129d7b25b6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:44.851621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-6b23019c-124b-4419-8fd9-e129d7b25b6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:44.853122 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1156/2351] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:14:44 2022] DELETE /identity/v3/projects/f0389d79f6bd420ba06f8a2b72e380a3 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:44.857255 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275eedf-745a-4421-9f94-0fa02dbd6377 service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:44.867501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275eedf-745a-4421-9f94-0fa02dbd6377 service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:44.925274 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38029b43-6af8-4f1f-b78b-d6798e0c5ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:44.925998 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38029b43-6af8-4f1f-b78b-d6798e0c5ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:44.926391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38029b43-6af8-4f1f-b78b-d6798e0c5ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:44.926693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38029b43-6af8-4f1f-b78b-d6798e0c5ff0 admin admin] PATH_INFO: `/v3/users/42f89d219cc34c5b9d24ebcb1cef7ce7` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:44.927839 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38029b43-6af8-4f1f-b78b-d6798e0c5ff0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42f89d219cc34c5b9d24ebcb1cef7ce7)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:44.929879 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38029b43-6af8-4f1f-b78b-d6798e0c5ff0 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:44.983764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-38029b43-6af8-4f1f-b78b-d6798e0c5ff0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42f89d219cc34c5b9d24ebcb1cef7ce7'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:44.991246 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-38029b43-6af8-4f1f-b78b-d6798e0c5ff0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42f89d219cc34c5b9d24ebcb1cef7ce7'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:44.998259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-38029b43-6af8-4f1f-b78b-d6798e0c5ff0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42f89d219cc34c5b9d24ebcb1cef7ce7'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:44.998874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-38029b43-6af8-4f1f-b78b-d6798e0c5ff0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42f89d219cc34c5b9d24ebcb1cef7ce7'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:44.999510 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-38029b43-6af8-4f1f-b78b-d6798e0c5ff0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42f89d219cc34c5b9d24ebcb1cef7ce7'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:45.016244 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1196/2352] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:14:44 2022] DELETE /identity/v3/users/42f89d219cc34c5b9d24ebcb1cef7ce7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:45.309764 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23019c-124b-4419-8fd9-e129d7b25b6f admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:45.311305 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6b23019c-124b-4419-8fd9-e129d7b25b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:45.325367 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:45.339268 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23019c-124b-4419-8fd9-e129d7b25b6f admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:45.411392 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5c2bea-15d0-4080-beeb-870036a520ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:45.412034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c2bea-15d0-4080-beeb-870036a520ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:45.412311 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c2bea-15d0-4080-beeb-870036a520ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:45.412580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c2bea-15d0-4080-beeb-870036a520ac admin admin] PATH_INFO: `/v3/projects/b74133d544994ab698fe82f7d8da2bd5` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:45.413734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c2bea-15d0-4080-beeb-870036a520ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=b74133d544994ab698fe82f7d8da2bd5)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:45.415844 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5c2bea-15d0-4080-beeb-870036a520ac admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:45.473939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-3a5c2bea-15d0-4080-beeb-870036a520ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b74133d544994ab698fe82f7d8da2bd5', 'request_id': 'req-3a5c2bea-15d0-4080-beeb-870036a520ac'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:45.480026 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-3a5c2bea-15d0-4080-beeb-870036a520ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b74133d544994ab698fe82f7d8da2bd5', 'request_id': 'req-3a5c2bea-15d0-4080-beeb-870036a520ac'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:45.485313 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-3a5c2bea-15d0-4080-beeb-870036a520ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b74133d544994ab698fe82f7d8da2bd5', 'request_id': 'req-3a5c2bea-15d0-4080-beeb-870036a520ac'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:45.497852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-3a5c2bea-15d0-4080-beeb-870036a520ac admin admin] The token cache is being invalidate because project b74133d544994ab698fe82f7d8da2bd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:14:45.498118 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-3a5c2bea-15d0-4080-beeb-870036a520ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:45.498725 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-3a5c2bea-15d0-4080-beeb-870036a520ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:14:45.500209 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1157/2353] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:14:45 2022] DELETE /identity/v3/projects/b74133d544994ab698fe82f7d8da2bd5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:45.505023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748be7ef-a9a8-47c8-9db3-a69f8dc1dc6d None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:45.505484 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748be7ef-a9a8-47c8-9db3-a69f8dc1dc6d None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:45.505608 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748be7ef-a9a8-47c8-9db3-a69f8dc1dc6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:45.524954 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-748be7ef-a9a8-47c8-9db3-a69f8dc1dc6d None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:45.526172 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-748be7ef-a9a8-47c8-9db3-a69f8dc1dc6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:45.530358 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:45.588330 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1197/2354] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:14:45.597766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a5d21-5ee2-4092-a67c-bbcb78f8861d None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:45.598047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a5d21-5ee2-4092-a67c-bbcb78f8861d None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:45.598320 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a5d21-5ee2-4092-a67c-bbcb78f8861d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:45.618511 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-6f5a5d21-5ee2-4092-a67c-bbcb78f8861d None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:45.619738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6f5a5d21-5ee2-4092-a67c-bbcb78f8861d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:45.627989 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1158/2355] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:14:45.632148 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748be7ef-a9a8-47c8-9db3-a69f8dc1dc6d None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:45.641988 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748be7ef-a9a8-47c8-9db3-a69f8dc1dc6d None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:45.646547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18e452-dffb-47bf-a393-1ee03b099c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:45.647193 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18e452-dffb-47bf-a393-1ee03b099c87 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:45.648209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18e452-dffb-47bf-a393-1ee03b099c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:45.648531 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18e452-dffb-47bf-a393-1ee03b099c87 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:45.649214 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18e452-dffb-47bf-a393-1ee03b099c87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:45.649528 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18e452-dffb-47bf-a393-1ee03b099c87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:45.651185 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18e452-dffb-47bf-a393-1ee03b099c87 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:45.663815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1198/2356] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:45.668032 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a5d21-5ee2-4092-a67c-bbcb78f8861d None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:45.677851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a5d21-5ee2-4092-a67c-bbcb78f8861d None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:45.683662 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ccc65-5053-4133-86d2-8000af95cb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:45.684257 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ccc65-5053-4133-86d2-8000af95cb03 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:45.684539 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ccc65-5053-4133-86d2-8000af95cb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:45.684813 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2ccc65-5053-4133-86d2-8000af95cb03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:45.685452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ccc65-5053-4133-86d2-8000af95cb03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:45.687369 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2ccc65-5053-4133-86d2-8000af95cb03 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:45.714406 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1159/2357] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:14:45 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:14:45.718615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18e452-dffb-47bf-a393-1ee03b099c87 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:45.728965 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18e452-dffb-47bf-a393-1ee03b099c87 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:45.786535 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932c190-95bc-41de-b184-b8a0632ac377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:45.787179 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932c190-95bc-41de-b184-b8a0632ac377 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:45.787463 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932c190-95bc-41de-b184-b8a0632ac377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:45.787736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932c190-95bc-41de-b184-b8a0632ac377 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:45.788369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932c190-95bc-41de-b184-b8a0632ac377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:45.790321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932c190-95bc-41de-b184-b8a0632ac377 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:45.812663 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1199/2358] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:14:45 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:14:45.816853 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ccc65-5053-4133-86d2-8000af95cb03 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:45.827854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2ccc65-5053-4133-86d2-8000af95cb03 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:45.833905 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3d142-1eca-4ddb-954b-012faa48b9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:45.834584 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3d142-1eca-4ddb-954b-012faa48b9ac admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:45.834866 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3d142-1eca-4ddb-954b-012faa48b9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:45.835176 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3d142-1eca-4ddb-954b-012faa48b9ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:45.836118 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3d142-1eca-4ddb-954b-012faa48b9ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:45.837706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3d142-1eca-4ddb-954b-012faa48b9ac admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:45.848282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1160/2359] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:14:45.852290 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932c190-95bc-41de-b184-b8a0632ac377 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:45.863004 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932c190-95bc-41de-b184-b8a0632ac377 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:45.867747 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270859cd-581e-4da8-bac2-c0d45d38c384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:45.868340 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270859cd-581e-4da8-bac2-c0d45d38c384 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:45.868643 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270859cd-581e-4da8-bac2-c0d45d38c384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:45.868922 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270859cd-581e-4da8-bac2-c0d45d38c384 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:45.869586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270859cd-581e-4da8-bac2-c0d45d38c384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:45.871365 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270859cd-581e-4da8-bac2-c0d45d38c384 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:45.882896 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1200/2360] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:14:45.887002 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3d142-1eca-4ddb-954b-012faa48b9ac admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:45.897215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3d142-1eca-4ddb-954b-012faa48b9ac admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:45.901744 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c93fc4-5577-4600-ba87-45a4ee3fac58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:45.902379 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c93fc4-5577-4600-ba87-45a4ee3fac58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:45.902657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c93fc4-5577-4600-ba87-45a4ee3fac58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:45.902926 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c93fc4-5577-4600-ba87-45a4ee3fac58 admin admin] PATH_INFO: `/v3/projects/4a4b57a964b34acdb48e224ef374e22f/users/1506e684fb3b400992900f071a81b591/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:45.918691 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c93fc4-5577-4600-ba87-45a4ee3fac58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a4b57a964b34acdb48e224ef374e22f, user_id=1506e684fb3b400992900f071a81b591, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:45.921336 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c93fc4-5577-4600-ba87-45a4ee3fac58 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:45.934234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1161/2361] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:14:45 2022] PUT /identity/v3/projects/4a4b57a964b34acdb48e224ef374e22f/users/1506e684fb3b400992900f071a81b591/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:45.944510 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1dac0-db32-47bf-84cb-3675fc2068bc None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:45.944802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1dac0-db32-47bf-84cb-3675fc2068bc None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:45.945081 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1dac0-db32-47bf-84cb-3675fc2068bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:45.977739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-09c1dac0-db32-47bf-84cb-3675fc2068bc None None] MFA Rules not processed for user `1506e684fb3b400992900f071a81b591`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:45.978966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-09c1dac0-db32-47bf-84cb-3675fc2068bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:46.044621 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1201/2362] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:14:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:14:46.049614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c93fc4-5577-4600-ba87-45a4ee3fac58 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:46.085207 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:46.117871 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c93fc4-5577-4600-ba87-45a4ee3fac58 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:46.123249 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fddd4e9-3a1a-400b-a40a-8b83ec8a4c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:46.123933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddd4e9-3a1a-400b-a40a-8b83ec8a4c3e admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:46.124280 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddd4e9-3a1a-400b-a40a-8b83ec8a4c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:46.124594 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddd4e9-3a1a-400b-a40a-8b83ec8a4c3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.125325 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fddd4e9-3a1a-400b-a40a-8b83ec8a4c3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:46.127698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fddd4e9-3a1a-400b-a40a-8b83ec8a4c3e admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:46.156553 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1162/2363] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:14:46 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:14:46.161453 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1dac0-db32-47bf-84cb-3675fc2068bc None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:46.175768 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:46.177176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1dac0-db32-47bf-84cb-3675fc2068bc None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:46.238673 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7e8c6-f97c-4b17-b821-d5f343d3956b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:46.239341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7e8c6-f97c-4b17-b821-d5f343d3956b admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:46.239622 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7e8c6-f97c-4b17-b821-d5f343d3956b admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:46.239896 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce7e8c6-f97c-4b17-b821-d5f343d3956b admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.240519 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7e8c6-f97c-4b17-b821-d5f343d3956b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:46.242451 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce7e8c6-f97c-4b17-b821-d5f343d3956b admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:46.267145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1202/2364] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:14:46 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:14:46.269729 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fddd4e9-3a1a-400b-a40a-8b83ec8a4c3e admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:46.280324 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fddd4e9-3a1a-400b-a40a-8b83ec8a4c3e admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:46.286714 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57282cc4-9dcd-4c5d-9991-0a5bf4acd1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:46.287468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57282cc4-9dcd-4c5d-9991-0a5bf4acd1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:46.287752 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57282cc4-9dcd-4c5d-9991-0a5bf4acd1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:46.288168 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57282cc4-9dcd-4c5d-9991-0a5bf4acd1d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.288832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57282cc4-9dcd-4c5d-9991-0a5bf4acd1d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:46.290809 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57282cc4-9dcd-4c5d-9991-0a5bf4acd1d1 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:46.304040 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1163/2365] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:14:46.308069 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7e8c6-f97c-4b17-b821-d5f343d3956b admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:46.318673 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce7e8c6-f97c-4b17-b821-d5f343d3956b admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:46.323806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20a1c3-ea56-4749-bd74-cee5f4197767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:46.324466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20a1c3-ea56-4749-bd74-cee5f4197767 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:46.324745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20a1c3-ea56-4749-bd74-cee5f4197767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:46.325018 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20a1c3-ea56-4749-bd74-cee5f4197767 admin admin] PATH_INFO: `/v3/projects/ac06792565f64c7ba6315609e2d59543/users/199c8bc744404256a55c0b78971900ea/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.342187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20a1c3-ea56-4749-bd74-cee5f4197767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac06792565f64c7ba6315609e2d59543, user_id=199c8bc744404256a55c0b78971900ea, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:46.344925 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20a1c3-ea56-4749-bd74-cee5f4197767 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:46.364062 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1203/2366] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:14:46 2022] PUT /identity/v3/projects/ac06792565f64c7ba6315609e2d59543/users/199c8bc744404256a55c0b78971900ea/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:46.368054 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57282cc4-9dcd-4c5d-9991-0a5bf4acd1d1 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:46.379840 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57282cc4-9dcd-4c5d-9991-0a5bf4acd1d1 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:46.435733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321eb889-fde4-434d-8cca-599b175f00f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:46.436342 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321eb889-fde4-434d-8cca-599b175f00f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:46.436620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321eb889-fde4-434d-8cca-599b175f00f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:46.436889 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321eb889-fde4-434d-8cca-599b175f00f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.437527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321eb889-fde4-434d-8cca-599b175f00f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:46.439114 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321eb889-fde4-434d-8cca-599b175f00f6 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:46.449356 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1164/2367] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:14:46.453616 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20a1c3-ea56-4749-bd74-cee5f4197767 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:46.464763 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20a1c3-ea56-4749-bd74-cee5f4197767 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:46.469330 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737fe140-324c-4861-84f6-2d25368353ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:46.469967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fe140-324c-4861-84f6-2d25368353ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:46.470244 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fe140-324c-4861-84f6-2d25368353ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:46.471920 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fe140-324c-4861-84f6-2d25368353ba admin admin] PATH_INFO: `/v3/projects/ac06792565f64c7ba6315609e2d59543/users/199c8bc744404256a55c0b78971900ea/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.474000 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737fe140-324c-4861-84f6-2d25368353ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac06792565f64c7ba6315609e2d59543, user_id=199c8bc744404256a55c0b78971900ea, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:46.476750 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737fe140-324c-4861-84f6-2d25368353ba admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:46.488553 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1204/2368] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:14:46 2022] PUT /identity/v3/projects/ac06792565f64c7ba6315609e2d59543/users/199c8bc744404256a55c0b78971900ea/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:46.498872 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493e2c6-5462-43f8-96ea-169cdfebad8a None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:46.499249 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493e2c6-5462-43f8-96ea-169cdfebad8a None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:46.499529 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493e2c6-5462-43f8-96ea-169cdfebad8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.529078 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-9493e2c6-5462-43f8-96ea-169cdfebad8a None None] MFA Rules not processed for user `199c8bc744404256a55c0b78971900ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:46.530366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-9493e2c6-5462-43f8-96ea-169cdfebad8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:46.604443 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1165/2369] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:14:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:14:46.615833 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737fe140-324c-4861-84f6-2d25368353ba admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:46.617605 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-737fe140-324c-4861-84f6-2d25368353ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:46.681558 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737fe140-324c-4861-84f6-2d25368353ba admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:46.685160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b81fe-6d4c-415b-bb63-5440760281a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:46.685755 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b81fe-6d4c-415b-bb63-5440760281a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:46.686031 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b81fe-6d4c-415b-bb63-5440760281a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:46.686299 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b81fe-6d4c-415b-bb63-5440760281a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.743207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b81fe-6d4c-415b-bb63-5440760281a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:46.744658 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b81fe-6d4c-415b-bb63-5440760281a5 service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:46.754621 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1205/2370] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:14:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:14:52.402534 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493e2c6-5462-43f8-96ea-169cdfebad8a None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:52.404087 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-9493e2c6-5462-43f8-96ea-169cdfebad8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:52.416596 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493e2c6-5462-43f8-96ea-169cdfebad8a None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:52.420263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667790b1-c517-47b7-8c05-3b63bc80e50f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:52.420972 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667790b1-c517-47b7-8c05-3b63bc80e50f service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:52.421317 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667790b1-c517-47b7-8c05-3b63bc80e50f service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:52.421670 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667790b1-c517-47b7-8c05-3b63bc80e50f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:52.431751 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667790b1-c517-47b7-8c05-3b63bc80e50f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:52.433302 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667790b1-c517-47b7-8c05-3b63bc80e50f service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:52.444521 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1166/2371] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:14:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:14:56.749494 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=90,keystone:INSERT=5 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:02.439821 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=93,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:07.920990 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00033de8-d0a3-4f8e-b226-9364559366f6 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:07.921456 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00033de8-d0a3-4f8e-b226-9364559366f6 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:07.921587 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00033de8-d0a3-4f8e-b226-9364559366f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:07.940865 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-00033de8-d0a3-4f8e-b226-9364559366f6 None None] MFA Rules not processed for user `76ebecce283f491e82a961d7f739878f`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:07.942000 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-00033de8-d0a3-4f8e-b226-9364559366f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:07.997153 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1206/2372] 10.222.0.36 () {58 vars in 1009 bytes} [Fri Sep 2 09:15:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:08.047935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667790b1-c517-47b7-8c05-3b63bc80e50f service cinder] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:08.050414 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-667790b1-c517-47b7-8c05-3b63bc80e50f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:08.120118 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667790b1-c517-47b7-8c05-3b63bc80e50f service cinder] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:08.125481 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73968aeb-8f9f-490e-8c98-6dc19644f4cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92318948dcf1407b9fc8aa5c199787ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:08.126325 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73968aeb-8f9f-490e-8c98-6dc19644f4cc service swift] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:08.126664 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73968aeb-8f9f-490e-8c98-6dc19644f4cc service swift] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:08.127436 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73968aeb-8f9f-490e-8c98-6dc19644f4cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:08.138016 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73968aeb-8f9f-490e-8c98-6dc19644f4cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:08.138016 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73968aeb-8f9f-490e-8c98-6dc19644f4cc service swift] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:08.146786 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1167/2373] 10.222.0.36 () {60 vars in 1384 bytes} [Fri Sep 2 09:15:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:15:17.654315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00033de8-d0a3-4f8e-b226-9364559366f6 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:17.716833 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00033de8-d0a3-4f8e-b226-9364559366f6 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:17.721930 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c479c-eaa6-4ad7-bb46-73bdcd88da8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:17.722558 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c479c-eaa6-4ad7-bb46-73bdcd88da8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:17.722835 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c479c-eaa6-4ad7-bb46-73bdcd88da8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:17.723131 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9c479c-eaa6-4ad7-bb46-73bdcd88da8d admin admin] PATH_INFO: `/v3/users/1506e684fb3b400992900f071a81b591` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:17.724236 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c479c-eaa6-4ad7-bb46-73bdcd88da8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1506e684fb3b400992900f071a81b591)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:17.726333 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9c479c-eaa6-4ad7-bb46-73bdcd88da8d admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:17.772771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-0f9c479c-eaa6-4ad7-bb46-73bdcd88da8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1506e684fb3b400992900f071a81b591'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:17.781240 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-0f9c479c-eaa6-4ad7-bb46-73bdcd88da8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1506e684fb3b400992900f071a81b591'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:17.789344 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-0f9c479c-eaa6-4ad7-bb46-73bdcd88da8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1506e684fb3b400992900f071a81b591'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:17.789968 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-0f9c479c-eaa6-4ad7-bb46-73bdcd88da8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1506e684fb3b400992900f071a81b591'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:17.790607 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-0f9c479c-eaa6-4ad7-bb46-73bdcd88da8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1506e684fb3b400992900f071a81b591'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:17.808697 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1207/2374] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:15:17 2022] DELETE /identity/v3/users/1506e684fb3b400992900f071a81b591 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:17.813318 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111efaf-0f97-4432-ba1b-45a0ce0b51f2 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:17.813786 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111efaf-0f97-4432-ba1b-45a0ce0b51f2 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:17.813905 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c111efaf-0f97-4432-ba1b-45a0ce0b51f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:17.832753 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-c111efaf-0f97-4432-ba1b-45a0ce0b51f2 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:17.833950 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c111efaf-0f97-4432-ba1b-45a0ce0b51f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:17.890983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1168/2375] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:17.900657 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c479c-eaa6-4ad7-bb46-73bdcd88da8d admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:17.902061 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0f9c479c-eaa6-4ad7-bb46-73bdcd88da8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:17.930749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:17.961680 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9c479c-eaa6-4ad7-bb46-73bdcd88da8d admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:17.965551 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8857df1d-64b9-4f5d-83a7-42b8b2540044 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:17.966158 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857df1d-64b9-4f5d-83a7-42b8b2540044 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:17.966437 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857df1d-64b9-4f5d-83a7-42b8b2540044 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:17.966710 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8857df1d-64b9-4f5d-83a7-42b8b2540044 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:17.978630 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8857df1d-64b9-4f5d-83a7-42b8b2540044 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:17.980342 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8857df1d-64b9-4f5d-83a7-42b8b2540044 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:17.992691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1208/2376] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:15:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:15:18.364263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111efaf-0f97-4432-ba1b-45a0ce0b51f2 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:18.365722 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c111efaf-0f97-4432-ba1b-45a0ce0b51f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:18.377624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c111efaf-0f97-4432-ba1b-45a0ce0b51f2 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:18.382210 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91a1db-9a2b-47a1-8a58-d2cfc965b6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:18.382840 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91a1db-9a2b-47a1-8a58-d2cfc965b6b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:18.383146 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91a1db-9a2b-47a1-8a58-d2cfc965b6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:18.383421 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f91a1db-9a2b-47a1-8a58-d2cfc965b6b5 admin admin] PATH_INFO: `/v3/projects/4a4b57a964b34acdb48e224ef374e22f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:18.384537 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f91a1db-9a2b-47a1-8a58-d2cfc965b6b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a4b57a964b34acdb48e224ef374e22f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:18.386421 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f91a1db-9a2b-47a1-8a58-d2cfc965b6b5 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:18.444745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-0f91a1db-9a2b-47a1-8a58-d2cfc965b6b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a4b57a964b34acdb48e224ef374e22f', 'request_id': 'req-0f91a1db-9a2b-47a1-8a58-d2cfc965b6b5'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:18.451565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-0f91a1db-9a2b-47a1-8a58-d2cfc965b6b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a4b57a964b34acdb48e224ef374e22f', 'request_id': 'req-0f91a1db-9a2b-47a1-8a58-d2cfc965b6b5'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:18.464565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-0f91a1db-9a2b-47a1-8a58-d2cfc965b6b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a4b57a964b34acdb48e224ef374e22f', 'request_id': 'req-0f91a1db-9a2b-47a1-8a58-d2cfc965b6b5'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:18.480018 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-0f91a1db-9a2b-47a1-8a58-d2cfc965b6b5 admin admin] The token cache is being invalidate because project 4a4b57a964b34acdb48e224ef374e22f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:15:18.480538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-0f91a1db-9a2b-47a1-8a58-d2cfc965b6b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:18.481414 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-0f91a1db-9a2b-47a1-8a58-d2cfc965b6b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:18.483639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1169/2377] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:15:18 2022] DELETE /identity/v3/projects/4a4b57a964b34acdb48e224ef374e22f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:18.487771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8857df1d-64b9-4f5d-83a7-42b8b2540044 service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:18.498001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8857df1d-64b9-4f5d-83a7-42b8b2540044 service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:18.562141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36729d6d-bde6-4379-b9fa-81394b5ac0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:18.562791 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36729d6d-bde6-4379-b9fa-81394b5ac0ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:18.563096 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36729d6d-bde6-4379-b9fa-81394b5ac0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:18.563360 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36729d6d-bde6-4379-b9fa-81394b5ac0ac admin admin] PATH_INFO: `/v3/users/199c8bc744404256a55c0b78971900ea` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:18.565024 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36729d6d-bde6-4379-b9fa-81394b5ac0ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=199c8bc744404256a55c0b78971900ea)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:18.566944 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36729d6d-bde6-4379-b9fa-81394b5ac0ac admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:18.618724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-36729d6d-bde6-4379-b9fa-81394b5ac0ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '199c8bc744404256a55c0b78971900ea'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:18.626227 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-36729d6d-bde6-4379-b9fa-81394b5ac0ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '199c8bc744404256a55c0b78971900ea'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:18.635495 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-36729d6d-bde6-4379-b9fa-81394b5ac0ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '199c8bc744404256a55c0b78971900ea'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:18.636137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-36729d6d-bde6-4379-b9fa-81394b5ac0ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '199c8bc744404256a55c0b78971900ea'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:18.636785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-36729d6d-bde6-4379-b9fa-81394b5ac0ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '199c8bc744404256a55c0b78971900ea'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:18.654809 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1209/2378] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:15:18 2022] DELETE /identity/v3/users/199c8bc744404256a55c0b78971900ea => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:15:18.951498 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91a1db-9a2b-47a1-8a58-d2cfc965b6b5 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:18.953049 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-0f91a1db-9a2b-47a1-8a58-d2cfc965b6b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:18.979479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f91a1db-9a2b-47a1-8a58-d2cfc965b6b5 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:18.991529 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:19.043997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ded239-6c0d-4c47-9185-d95a6402a74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:19.044712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ded239-6c0d-4c47-9185-d95a6402a74f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.045115 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ded239-6c0d-4c47-9185-d95a6402a74f admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.045458 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ded239-6c0d-4c47-9185-d95a6402a74f admin admin] PATH_INFO: `/v3/projects/ac06792565f64c7ba6315609e2d59543` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.046674 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ded239-6c0d-4c47-9185-d95a6402a74f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac06792565f64c7ba6315609e2d59543)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:19.048708 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ded239-6c0d-4c47-9185-d95a6402a74f admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:19.098338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-48ded239-6c0d-4c47-9185-d95a6402a74f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac06792565f64c7ba6315609e2d59543', 'request_id': 'req-48ded239-6c0d-4c47-9185-d95a6402a74f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:19.108102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-48ded239-6c0d-4c47-9185-d95a6402a74f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac06792565f64c7ba6315609e2d59543', 'request_id': 'req-48ded239-6c0d-4c47-9185-d95a6402a74f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:19.116204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-48ded239-6c0d-4c47-9185-d95a6402a74f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac06792565f64c7ba6315609e2d59543', 'request_id': 'req-48ded239-6c0d-4c47-9185-d95a6402a74f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:19.130142 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-48ded239-6c0d-4c47-9185-d95a6402a74f admin admin] The token cache is being invalidate because project ac06792565f64c7ba6315609e2d59543 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:15:19.130434 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-48ded239-6c0d-4c47-9185-d95a6402a74f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:19.131062 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-48ded239-6c0d-4c47-9185-d95a6402a74f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:19.132830 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1170/2379] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:15:18 2022] DELETE /identity/v3/projects/ac06792565f64c7ba6315609e2d59543 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:19.145598 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f0c2f-a9b9-4cd3-bf06-a0aad3904df5 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.145885 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f0c2f-a9b9-4cd3-bf06-a0aad3904df5 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.146162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f0c2f-a9b9-4cd3-bf06-a0aad3904df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.165384 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-6c2f0c2f-a9b9-4cd3-bf06-a0aad3904df5 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:19.166568 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6c2f0c2f-a9b9-4cd3-bf06-a0aad3904df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:19.207461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:19.225917 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1210/2380] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:19.233584 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb737a-5fc8-4d17-880d-822aa6cd9d62 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.233873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb737a-5fc8-4d17-880d-822aa6cd9d62 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.234153 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb737a-5fc8-4d17-880d-822aa6cd9d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.255486 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-f1eb737a-5fc8-4d17-880d-822aa6cd9d62 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:19.256676 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-f1eb737a-5fc8-4d17-880d-822aa6cd9d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:19.264403 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1171/2381] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:19.268468 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f0c2f-a9b9-4cd3-bf06-a0aad3904df5 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:19.277877 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f0c2f-a9b9-4cd3-bf06-a0aad3904df5 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:19.282613 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e7d34-7742-483e-8490-cac5a3e4f23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:19.283773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e7d34-7742-483e-8490-cac5a3e4f23e admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.284055 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e7d34-7742-483e-8490-cac5a3e4f23e admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.284328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e7d34-7742-483e-8490-cac5a3e4f23e admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.284970 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e7d34-7742-483e-8490-cac5a3e4f23e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:19.285276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e7d34-7742-483e-8490-cac5a3e4f23e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:19.287589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e7d34-7742-483e-8490-cac5a3e4f23e admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:19.299759 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1211/2382] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:19.303824 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb737a-5fc8-4d17-880d-822aa6cd9d62 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:19.312772 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb737a-5fc8-4d17-880d-822aa6cd9d62 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:19.317412 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480fade-0f81-41bd-9a7b-1bee260f1544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:19.317998 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480fade-0f81-41bd-9a7b-1bee260f1544 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.318275 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480fade-0f81-41bd-9a7b-1bee260f1544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.318545 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480fade-0f81-41bd-9a7b-1bee260f1544 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.319204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480fade-0f81-41bd-9a7b-1bee260f1544 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:19.321078 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480fade-0f81-41bd-9a7b-1bee260f1544 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:19.348691 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1172/2383] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:15:19 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:19.352609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e7d34-7742-483e-8490-cac5a3e4f23e admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:19.363608 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e7d34-7742-483e-8490-cac5a3e4f23e admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:19.419427 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965771bb-0a5b-4a15-a9b3-fea23fa52702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:19.419952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965771bb-0a5b-4a15-a9b3-fea23fa52702 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.420228 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965771bb-0a5b-4a15-a9b3-fea23fa52702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.420498 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965771bb-0a5b-4a15-a9b3-fea23fa52702 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.421124 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965771bb-0a5b-4a15-a9b3-fea23fa52702 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:19.423044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965771bb-0a5b-4a15-a9b3-fea23fa52702 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:19.445194 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1212/2384] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:15:19 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:19.450041 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480fade-0f81-41bd-9a7b-1bee260f1544 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:19.460632 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480fade-0f81-41bd-9a7b-1bee260f1544 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:19.465311 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7104f0-c991-47f6-a96f-078bd38c73e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:19.465904 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7104f0-c991-47f6-a96f-078bd38c73e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.466183 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7104f0-c991-47f6-a96f-078bd38c73e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.466453 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7104f0-c991-47f6-a96f-078bd38c73e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.467122 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7104f0-c991-47f6-a96f-078bd38c73e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:19.468706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7104f0-c991-47f6-a96f-078bd38c73e7 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:19.479380 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1173/2385] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:19.483224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965771bb-0a5b-4a15-a9b3-fea23fa52702 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:19.493709 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965771bb-0a5b-4a15-a9b3-fea23fa52702 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:19.498317 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2b1c1-ba53-4350-8acd-b04b7d073348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:19.498901 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2b1c1-ba53-4350-8acd-b04b7d073348 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.499270 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2b1c1-ba53-4350-8acd-b04b7d073348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.499513 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2b1c1-ba53-4350-8acd-b04b7d073348 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.500261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2b1c1-ba53-4350-8acd-b04b7d073348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:19.501845 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2b1c1-ba53-4350-8acd-b04b7d073348 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:19.513578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1213/2386] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:19.517515 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7104f0-c991-47f6-a96f-078bd38c73e7 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:19.527266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7104f0-c991-47f6-a96f-078bd38c73e7 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:19.531822 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869a3a0-6342-4950-b764-d5e4d0626035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:19.532462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869a3a0-6342-4950-b764-d5e4d0626035 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.532742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869a3a0-6342-4950-b764-d5e4d0626035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.533016 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869a3a0-6342-4950-b764-d5e4d0626035 admin admin] PATH_INFO: `/v3/projects/1362e65bec81409288daee0bcfb34e3f/users/818d9721208b4ee6a22c94cb637d7ec4/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.547703 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869a3a0-6342-4950-b764-d5e4d0626035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1362e65bec81409288daee0bcfb34e3f, user_id=818d9721208b4ee6a22c94cb637d7ec4, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:19.550906 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869a3a0-6342-4950-b764-d5e4d0626035 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:19.563442 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1174/2387] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:15:19 2022] PUT /identity/v3/projects/1362e65bec81409288daee0bcfb34e3f/users/818d9721208b4ee6a22c94cb637d7ec4/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:15:19.574660 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a609bd0-9f12-4b77-890a-b4b7d8fd1e08 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.574938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a609bd0-9f12-4b77-890a-b4b7d8fd1e08 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.575261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a609bd0-9f12-4b77-890a-b4b7d8fd1e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.603887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-8a609bd0-9f12-4b77-890a-b4b7d8fd1e08 None None] MFA Rules not processed for user `818d9721208b4ee6a22c94cb637d7ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:19.605092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8a609bd0-9f12-4b77-890a-b4b7d8fd1e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:19.667046 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1214/2388] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:15:19.671561 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869a3a0-6342-4950-b764-d5e4d0626035 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:19.712795 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:19.734549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869a3a0-6342-4950-b764-d5e4d0626035 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:19.739279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008efe9-6c08-4f33-b5aa-0ee81170851b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:19.739882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008efe9-6c08-4f33-b5aa-0ee81170851b admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.740163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008efe9-6c08-4f33-b5aa-0ee81170851b admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.740437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008efe9-6c08-4f33-b5aa-0ee81170851b admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.741059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008efe9-6c08-4f33-b5aa-0ee81170851b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:19.742941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008efe9-6c08-4f33-b5aa-0ee81170851b admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:19.769598 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1175/2389] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:15:19 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:19.774254 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a609bd0-9f12-4b77-890a-b4b7d8fd1e08 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:19.785625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a609bd0-9f12-4b77-890a-b4b7d8fd1e08 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:19.824882 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:19.850335 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c3239-4d58-4f54-b81a-a581a03d60d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:19.850949 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c3239-4d58-4f54-b81a-a581a03d60d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.851257 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c3239-4d58-4f54-b81a-a581a03d60d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.851527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c3239-4d58-4f54-b81a-a581a03d60d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.852163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c3239-4d58-4f54-b81a-a581a03d60d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:19.854074 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c3239-4d58-4f54-b81a-a581a03d60d6 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:19.877272 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1215/2390] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:15:19 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:19.881464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008efe9-6c08-4f33-b5aa-0ee81170851b admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:19.891920 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008efe9-6c08-4f33-b5aa-0ee81170851b admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:19.897560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f879f5-495a-4819-a3f1-0ba8fadba205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:19.898171 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f879f5-495a-4819-a3f1-0ba8fadba205 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.898468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f879f5-495a-4819-a3f1-0ba8fadba205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.898745 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f879f5-495a-4819-a3f1-0ba8fadba205 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.899420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f879f5-495a-4819-a3f1-0ba8fadba205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:19.901013 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f879f5-495a-4819-a3f1-0ba8fadba205 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:19.915595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1176/2391] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:19.916750 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c3239-4d58-4f54-b81a-a581a03d60d6 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:19.927220 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c3239-4d58-4f54-b81a-a581a03d60d6 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:19.932051 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d3b3c-be8c-4084-83cd-dacd41b75916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:19.932645 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d3b3c-be8c-4084-83cd-dacd41b75916 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.932922 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d3b3c-be8c-4084-83cd-dacd41b75916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.933175 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d3b3c-be8c-4084-83cd-dacd41b75916 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.933905 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d3b3c-be8c-4084-83cd-dacd41b75916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:19.935674 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d3b3c-be8c-4084-83cd-dacd41b75916 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:19.946414 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1216/2392] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:19.950749 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f879f5-495a-4819-a3f1-0ba8fadba205 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:19.962137 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f879f5-495a-4819-a3f1-0ba8fadba205 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:19.966855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f116c24f-afe5-4b03-b7eb-4ce13b5e142a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:19.967610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116c24f-afe5-4b03-b7eb-4ce13b5e142a admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.967816 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116c24f-afe5-4b03-b7eb-4ce13b5e142a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.968092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f116c24f-afe5-4b03-b7eb-4ce13b5e142a admin admin] PATH_INFO: `/v3/projects/8a92342e131a4302a25d57da5822ef28/users/f3388bf181d4462ead35818fec31af01/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.983045 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f116c24f-afe5-4b03-b7eb-4ce13b5e142a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a92342e131a4302a25d57da5822ef28, user_id=f3388bf181d4462ead35818fec31af01, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:19.985723 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f116c24f-afe5-4b03-b7eb-4ce13b5e142a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:19.998125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1177/2393] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:15:19 2022] PUT /identity/v3/projects/8a92342e131a4302a25d57da5822ef28/users/f3388bf181d4462ead35818fec31af01/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:20.009600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7d47d-2b0b-4059-8c3a-48fa8b81111d None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:20.009884 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7d47d-2b0b-4059-8c3a-48fa8b81111d None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:20.010156 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7d47d-2b0b-4059-8c3a-48fa8b81111d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:20.038194 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-a8f7d47d-2b0b-4059-8c3a-48fa8b81111d None None] MFA Rules not processed for user `f3388bf181d4462ead35818fec31af01`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:20.039409 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a8f7d47d-2b0b-4059-8c3a-48fa8b81111d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:20.101747 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1217/2394] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:20.106266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f116c24f-afe5-4b03-b7eb-4ce13b5e142a admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:20.167853 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f116c24f-afe5-4b03-b7eb-4ce13b5e142a admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:20.172496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227370df-87f2-4596-922d-64d5a7c59f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:20.173094 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227370df-87f2-4596-922d-64d5a7c59f61 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:20.173369 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227370df-87f2-4596-922d-64d5a7c59f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:20.173646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227370df-87f2-4596-922d-64d5a7c59f61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:20.174444 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227370df-87f2-4596-922d-64d5a7c59f61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:20.176415 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227370df-87f2-4596-922d-64d5a7c59f61 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:20.203895 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1178/2395] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:15:20 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:20.208642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7d47d-2b0b-4059-8c3a-48fa8b81111d None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:20.219319 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7d47d-2b0b-4059-8c3a-48fa8b81111d None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:20.276829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ead85-976b-4fe5-bfa1-6b9b74ca575c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:20.277630 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ead85-976b-4fe5-bfa1-6b9b74ca575c admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:20.277976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ead85-976b-4fe5-bfa1-6b9b74ca575c admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:20.278264 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ead85-976b-4fe5-bfa1-6b9b74ca575c admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:20.278900 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ead85-976b-4fe5-bfa1-6b9b74ca575c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:20.281016 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ead85-976b-4fe5-bfa1-6b9b74ca575c admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:20.301597 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1218/2396] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:15:20 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:20.305511 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227370df-87f2-4596-922d-64d5a7c59f61 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:20.316141 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227370df-87f2-4596-922d-64d5a7c59f61 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:20.320941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134c077-86e1-4ead-9509-52a0694df20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:20.321534 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134c077-86e1-4ead-9509-52a0694df20d admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:20.321817 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134c077-86e1-4ead-9509-52a0694df20d admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:20.322087 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b134c077-86e1-4ead-9509-52a0694df20d admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:20.322749 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134c077-86e1-4ead-9509-52a0694df20d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:20.324366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b134c077-86e1-4ead-9509-52a0694df20d admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:20.335707 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1179/2397] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:20.339684 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ead85-976b-4fe5-bfa1-6b9b74ca575c admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:20.349833 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ead85-976b-4fe5-bfa1-6b9b74ca575c admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:20.354735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be412c93-a0ca-4ce8-821c-1689746e4f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:20.355439 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be412c93-a0ca-4ce8-821c-1689746e4f26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:20.355723 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be412c93-a0ca-4ce8-821c-1689746e4f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:20.356003 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be412c93-a0ca-4ce8-821c-1689746e4f26 admin admin] PATH_INFO: `/v3/projects/ffd94c52e44b47f49a4b8cec1af4477c/users/4800cda910f44690870cee93ab353886/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:20.373857 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be412c93-a0ca-4ce8-821c-1689746e4f26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffd94c52e44b47f49a4b8cec1af4477c, user_id=4800cda910f44690870cee93ab353886, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:20.376638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be412c93-a0ca-4ce8-821c-1689746e4f26 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:20.389380 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1219/2398] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:15:20 2022] PUT /identity/v3/projects/ffd94c52e44b47f49a4b8cec1af4477c/users/4800cda910f44690870cee93ab353886/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:20.393696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134c077-86e1-4ead-9509-52a0694df20d admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:20.404204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b134c077-86e1-4ead-9509-52a0694df20d admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:20.460120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e270dc-a7d7-46cc-a77d-c8ac4663aa26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:20.460722 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e270dc-a7d7-46cc-a77d-c8ac4663aa26 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:20.460979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e270dc-a7d7-46cc-a77d-c8ac4663aa26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:20.461248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e270dc-a7d7-46cc-a77d-c8ac4663aa26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:20.461879 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e270dc-a7d7-46cc-a77d-c8ac4663aa26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:20.463561 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e270dc-a7d7-46cc-a77d-c8ac4663aa26 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:20.468428 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:20.474097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1180/2399] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:20.478204 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be412c93-a0ca-4ce8-821c-1689746e4f26 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:20.488714 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be412c93-a0ca-4ce8-821c-1689746e4f26 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:20.493379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04db731-6503-4904-81fc-642fbdfdd530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:20.494101 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04db731-6503-4904-81fc-642fbdfdd530 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:20.494374 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04db731-6503-4904-81fc-642fbdfdd530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:20.494642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04db731-6503-4904-81fc-642fbdfdd530 admin admin] PATH_INFO: `/v3/projects/ffd94c52e44b47f49a4b8cec1af4477c/users/4800cda910f44690870cee93ab353886/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:20.496604 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04db731-6503-4904-81fc-642fbdfdd530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffd94c52e44b47f49a4b8cec1af4477c, user_id=4800cda910f44690870cee93ab353886, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:20.499263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04db731-6503-4904-81fc-642fbdfdd530 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:20.512145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1220/2400] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:15:20 2022] PUT /identity/v3/projects/ffd94c52e44b47f49a4b8cec1af4477c/users/4800cda910f44690870cee93ab353886/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:20.522837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6faaff8-9294-4a02-87ef-152db94e2afb None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:20.523157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6faaff8-9294-4a02-87ef-152db94e2afb None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:20.523435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6faaff8-9294-4a02-87ef-152db94e2afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:20.551395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-a6faaff8-9294-4a02-87ef-152db94e2afb None None] MFA Rules not processed for user `4800cda910f44690870cee93ab353886`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:20.552707 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-a6faaff8-9294-4a02-87ef-152db94e2afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:20.621878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1181/2401] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:20 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:20.633473 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04db731-6503-4904-81fc-642fbdfdd530 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:20.635189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-b04db731-6503-4904-81fc-642fbdfdd530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:20.640459 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:20.694446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04db731-6503-4904-81fc-642fbdfdd530 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:20.698099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32029810-eb4e-43bc-8729-f19008b3b114 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:20.698701 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32029810-eb4e-43bc-8729-f19008b3b114 service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:20.698983 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32029810-eb4e-43bc-8729-f19008b3b114 service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:20.699296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32029810-eb4e-43bc-8729-f19008b3b114 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:20.708365 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32029810-eb4e-43bc-8729-f19008b3b114 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:20.709793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32029810-eb4e-43bc-8729-f19008b3b114 service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:20.719459 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1221/2402] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:15:20 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:15:21.271195 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6faaff8-9294-4a02-87ef-152db94e2afb None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:21.281043 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6faaff8-9294-4a02-87ef-152db94e2afb None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:21.285473 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6807b-73e1-4a9a-8e1c-6730b8f7ed81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:21.286073 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6807b-73e1-4a9a-8e1c-6730b8f7ed81 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:21.286347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6807b-73e1-4a9a-8e1c-6730b8f7ed81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:21.286614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6807b-73e1-4a9a-8e1c-6730b8f7ed81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:21.342875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6807b-73e1-4a9a-8e1c-6730b8f7ed81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:21.344341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6807b-73e1-4a9a-8e1c-6730b8f7ed81 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:21.353983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1182/2403] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:15:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:15:25.409042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32029810-eb4e-43bc-8729-f19008b3b114 service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:25.410742 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-32029810-eb4e-43bc-8729-f19008b3b114 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:25.429069 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32029810-eb4e-43bc-8729-f19008b3b114 service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:25.432988 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77455854-c2cd-4e9e-a134-614736887482 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:25.433584 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77455854-c2cd-4e9e-a134-614736887482 service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:25.433900 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77455854-c2cd-4e9e-a134-614736887482 service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:25.434181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77455854-c2cd-4e9e-a134-614736887482 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:25.494946 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77455854-c2cd-4e9e-a134-614736887482 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:25.496426 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77455854-c2cd-4e9e-a134-614736887482 service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:25.506079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1222/2404] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:15:25 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:15:27.119283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0b86a-161f-4474-8f14-c2c63705182c None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:27.120306 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0b86a-161f-4474-8f14-c2c63705182c None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:27.123476 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0b86a-161f-4474-8f14-c2c63705182c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:27.160545 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-7ae0b86a-161f-4474-8f14-c2c63705182c None None] MFA Rules not processed for user `76ebecce283f491e82a961d7f739878f`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:27.161877 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7ae0b86a-161f-4474-8f14-c2c63705182c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:27.243121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1183/2405] 10.222.0.36 () {58 vars in 1009 bytes} [Fri Sep 2 09:15:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:27.251170 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77455854-c2cd-4e9e-a134-614736887482 service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:27.256688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-77455854-c2cd-4e9e-a134-614736887482 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:27.358795 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77455854-c2cd-4e9e-a134-614736887482 service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:27.363915 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e6094-9176-4f4d-9d10-f67085c0bbc1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92318948dcf1407b9fc8aa5c199787ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:27.364768 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e6094-9176-4f4d-9d10-f67085c0bbc1 service swift] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:27.365163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e6094-9176-4f4d-9d10-f67085c0bbc1 service swift] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:27.365437 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e6094-9176-4f4d-9d10-f67085c0bbc1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:27.374578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e6094-9176-4f4d-9d10-f67085c0bbc1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:27.376408 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852e6094-9176-4f4d-9d10-f67085c0bbc1 service swift] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:27.385176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1223/2406] 10.222.0.36 () {60 vars in 1384 bytes} [Fri Sep 2 09:15:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:15:27.915057 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0b86a-161f-4474-8f14-c2c63705182c None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:27.985517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:28.028981 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0b86a-161f-4474-8f14-c2c63705182c None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:28.034939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f28d0-f25f-48b8-97de-c52c293d3734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:28.035809 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f28d0-f25f-48b8-97de-c52c293d3734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:28.036334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f28d0-f25f-48b8-97de-c52c293d3734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:28.036721 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f28d0-f25f-48b8-97de-c52c293d3734 admin admin] PATH_INFO: `/v3/users/818d9721208b4ee6a22c94cb637d7ec4` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:28.038297 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f28d0-f25f-48b8-97de-c52c293d3734 admin admin] RBAC: Authorizing `identity:delete_user(user_id=818d9721208b4ee6a22c94cb637d7ec4)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:28.040641 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f28d0-f25f-48b8-97de-c52c293d3734 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:28.114546 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-407f28d0-f25f-48b8-97de-c52c293d3734 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '818d9721208b4ee6a22c94cb637d7ec4'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:28.115730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-407f28d0-f25f-48b8-97de-c52c293d3734 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '818d9721208b4ee6a22c94cb637d7ec4'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:28.134869 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-407f28d0-f25f-48b8-97de-c52c293d3734 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '818d9721208b4ee6a22c94cb637d7ec4'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:28.146449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-407f28d0-f25f-48b8-97de-c52c293d3734 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '818d9721208b4ee6a22c94cb637d7ec4'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:28.147488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-407f28d0-f25f-48b8-97de-c52c293d3734 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '818d9721208b4ee6a22c94cb637d7ec4'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:28.170752 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1184/2407] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:15:27 2022] DELETE /identity/v3/users/818d9721208b4ee6a22c94cb637d7ec4 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:28.175746 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a784a4-ee47-4158-999e-3b1187e97198 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:28.176496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a784a4-ee47-4158-999e-3b1187e97198 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:28.177550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a784a4-ee47-4158-999e-3b1187e97198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:28.224093 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-f9a784a4-ee47-4158-999e-3b1187e97198 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:28.229859 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f9a784a4-ee47-4158-999e-3b1187e97198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:28.303668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:28.381919 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1224/2408] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:15:28.391171 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f28d0-f25f-48b8-97de-c52c293d3734 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:28.393214 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-407f28d0-f25f-48b8-97de-c52c293d3734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:28.507139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f28d0-f25f-48b8-97de-c52c293d3734 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:28.510832 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248ed5f-e004-420f-9db3-f23167785447 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:28.511486 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248ed5f-e004-420f-9db3-f23167785447 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:28.511760 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248ed5f-e004-420f-9db3-f23167785447 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:28.512029 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248ed5f-e004-420f-9db3-f23167785447 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:28.522852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4248ed5f-e004-420f-9db3-f23167785447 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:28.524351 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4248ed5f-e004-420f-9db3-f23167785447 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:28.540046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1185/2409] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:15:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:15:28.986418 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a784a4-ee47-4158-999e-3b1187e97198 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:28.988077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f9a784a4-ee47-4158-999e-3b1187e97198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:29.003877 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a784a4-ee47-4158-999e-3b1187e97198 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:29.021342 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72bd922-7252-44ca-af0a-9601f038090d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:29.023143 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bd922-7252-44ca-af0a-9601f038090d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:29.023435 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bd922-7252-44ca-af0a-9601f038090d admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:29.023715 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72bd922-7252-44ca-af0a-9601f038090d admin admin] PATH_INFO: `/v3/projects/1362e65bec81409288daee0bcfb34e3f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:29.035023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72bd922-7252-44ca-af0a-9601f038090d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1362e65bec81409288daee0bcfb34e3f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:29.037178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72bd922-7252-44ca-af0a-9601f038090d admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:29.156130 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-a72bd922-7252-44ca-af0a-9601f038090d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1362e65bec81409288daee0bcfb34e3f', 'request_id': 'req-a72bd922-7252-44ca-af0a-9601f038090d'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:29.169404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-a72bd922-7252-44ca-af0a-9601f038090d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1362e65bec81409288daee0bcfb34e3f', 'request_id': 'req-a72bd922-7252-44ca-af0a-9601f038090d'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:29.191335 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-a72bd922-7252-44ca-af0a-9601f038090d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1362e65bec81409288daee0bcfb34e3f', 'request_id': 'req-a72bd922-7252-44ca-af0a-9601f038090d'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:29.215957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-a72bd922-7252-44ca-af0a-9601f038090d admin admin] The token cache is being invalidate because project 1362e65bec81409288daee0bcfb34e3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:15:29.216366 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-a72bd922-7252-44ca-af0a-9601f038090d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:29.217710 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-a72bd922-7252-44ca-af0a-9601f038090d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:29.219627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1225/2410] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:15:28 2022] DELETE /identity/v3/projects/1362e65bec81409288daee0bcfb34e3f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:29.224083 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248ed5f-e004-420f-9db3-f23167785447 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:29.235789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248ed5f-e004-420f-9db3-f23167785447 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:29.311576 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:29.324319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc3ce4-e2c4-4533-b602-200e11913069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:29.324960 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc3ce4-e2c4-4533-b602-200e11913069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:29.325244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc3ce4-e2c4-4533-b602-200e11913069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:29.325520 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc3ce4-e2c4-4533-b602-200e11913069 admin admin] PATH_INFO: `/v3/users/f3388bf181d4462ead35818fec31af01` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:29.326744 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc3ce4-e2c4-4533-b602-200e11913069 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3388bf181d4462ead35818fec31af01)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:29.333903 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc3ce4-e2c4-4533-b602-200e11913069 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:29.417071 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-80cc3ce4-e2c4-4533-b602-200e11913069 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3388bf181d4462ead35818fec31af01'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:29.419105 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-80cc3ce4-e2c4-4533-b602-200e11913069 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3388bf181d4462ead35818fec31af01'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:29.447637 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-80cc3ce4-e2c4-4533-b602-200e11913069 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3388bf181d4462ead35818fec31af01'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:29.465447 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-80cc3ce4-e2c4-4533-b602-200e11913069 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3388bf181d4462ead35818fec31af01'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:29.466274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-80cc3ce4-e2c4-4533-b602-200e11913069 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3388bf181d4462ead35818fec31af01'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:29.486003 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1186/2411] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:15:29 2022] DELETE /identity/v3/users/f3388bf181d4462ead35818fec31af01 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:29.817966 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72bd922-7252-44ca-af0a-9601f038090d admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:29.819600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a72bd922-7252-44ca-af0a-9601f038090d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:29.848850 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72bd922-7252-44ca-af0a-9601f038090d admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:29.923422 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e465a17-4538-4572-ba3f-9a8be4b48728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:29.924350 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e465a17-4538-4572-ba3f-9a8be4b48728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:29.924659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e465a17-4538-4572-ba3f-9a8be4b48728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:29.924927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e465a17-4538-4572-ba3f-9a8be4b48728 admin admin] PATH_INFO: `/v3/projects/8a92342e131a4302a25d57da5822ef28` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:29.926433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e465a17-4538-4572-ba3f-9a8be4b48728 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a92342e131a4302a25d57da5822ef28)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:29.928660 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e465a17-4538-4572-ba3f-9a8be4b48728 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:29.988916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-3e465a17-4538-4572-ba3f-9a8be4b48728 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a92342e131a4302a25d57da5822ef28', 'request_id': 'req-3e465a17-4538-4572-ba3f-9a8be4b48728'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:29.999217 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-3e465a17-4538-4572-ba3f-9a8be4b48728 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a92342e131a4302a25d57da5822ef28', 'request_id': 'req-3e465a17-4538-4572-ba3f-9a8be4b48728'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:30.003099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:30.007997 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-3e465a17-4538-4572-ba3f-9a8be4b48728 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a92342e131a4302a25d57da5822ef28', 'request_id': 'req-3e465a17-4538-4572-ba3f-9a8be4b48728'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:30.030646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-3e465a17-4538-4572-ba3f-9a8be4b48728 admin admin] The token cache is being invalidate because project 8a92342e131a4302a25d57da5822ef28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:15:30.031272 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-3e465a17-4538-4572-ba3f-9a8be4b48728 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:30.032403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-3e465a17-4538-4572-ba3f-9a8be4b48728 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:30.034483 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1226/2412] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:15:29 2022] DELETE /identity/v3/projects/8a92342e131a4302a25d57da5822ef28 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:30.038484 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc3ce4-e2c4-4533-b602-200e11913069 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:30.040024 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-80cc3ce4-e2c4-4533-b602-200e11913069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:30.054362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc3ce4-e2c4-4533-b602-200e11913069 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:30.114078 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44e474-1ae2-444a-a43a-ae1a2f917ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:30.114740 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44e474-1ae2-444a-a43a-ae1a2f917ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:30.115096 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44e474-1ae2-444a-a43a-ae1a2f917ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:30.115493 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44e474-1ae2-444a-a43a-ae1a2f917ee8 admin admin] PATH_INFO: `/v3/users/4800cda910f44690870cee93ab353886` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:30.116968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf44e474-1ae2-444a-a43a-ae1a2f917ee8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4800cda910f44690870cee93ab353886)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:30.118950 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf44e474-1ae2-444a-a43a-ae1a2f917ee8 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:30.169739 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-cf44e474-1ae2-444a-a43a-ae1a2f917ee8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4800cda910f44690870cee93ab353886'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:30.171181 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-cf44e474-1ae2-444a-a43a-ae1a2f917ee8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4800cda910f44690870cee93ab353886'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:30.184331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-cf44e474-1ae2-444a-a43a-ae1a2f917ee8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4800cda910f44690870cee93ab353886'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:30.190885 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:30.192940 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-cf44e474-1ae2-444a-a43a-ae1a2f917ee8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4800cda910f44690870cee93ab353886'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:30.193820 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-cf44e474-1ae2-444a-a43a-ae1a2f917ee8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4800cda910f44690870cee93ab353886'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:30.215207 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1187/2413] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:15:30 2022] DELETE /identity/v3/users/4800cda910f44690870cee93ab353886 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:15:30.519121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e465a17-4538-4572-ba3f-9a8be4b48728 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:30.520759 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-3e465a17-4538-4572-ba3f-9a8be4b48728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:30.545366 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e465a17-4538-4572-ba3f-9a8be4b48728 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:30.628393 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfbce4-c59a-4a1c-98dc-8bacec1f4930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:30.629040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfbce4-c59a-4a1c-98dc-8bacec1f4930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:30.629321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfbce4-c59a-4a1c-98dc-8bacec1f4930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:30.629654 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfbce4-c59a-4a1c-98dc-8bacec1f4930 admin admin] PATH_INFO: `/v3/projects/ffd94c52e44b47f49a4b8cec1af4477c` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:30.630802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbfbce4-c59a-4a1c-98dc-8bacec1f4930 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffd94c52e44b47f49a4b8cec1af4477c)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:30.633317 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbfbce4-c59a-4a1c-98dc-8bacec1f4930 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:30.690965 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-4fbfbce4-c59a-4a1c-98dc-8bacec1f4930 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffd94c52e44b47f49a4b8cec1af4477c', 'request_id': 'req-4fbfbce4-c59a-4a1c-98dc-8bacec1f4930'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:30.700687 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-4fbfbce4-c59a-4a1c-98dc-8bacec1f4930 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffd94c52e44b47f49a4b8cec1af4477c', 'request_id': 'req-4fbfbce4-c59a-4a1c-98dc-8bacec1f4930'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:30.707063 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-4fbfbce4-c59a-4a1c-98dc-8bacec1f4930 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffd94c52e44b47f49a4b8cec1af4477c', 'request_id': 'req-4fbfbce4-c59a-4a1c-98dc-8bacec1f4930'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:30.719190 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-4fbfbce4-c59a-4a1c-98dc-8bacec1f4930 admin admin] The token cache is being invalidate because project ffd94c52e44b47f49a4b8cec1af4477c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:15:30.719517 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-4fbfbce4-c59a-4a1c-98dc-8bacec1f4930 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:30.720205 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-4fbfbce4-c59a-4a1c-98dc-8bacec1f4930 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:30.724949 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1227/2414] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:15:30 2022] DELETE /identity/v3/projects/ffd94c52e44b47f49a4b8cec1af4477c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:30.730914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752408d-5aac-4ab4-8f11-af445bce8b49 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:30.731417 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752408d-5aac-4ab4-8f11-af445bce8b49 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:30.731529 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752408d-5aac-4ab4-8f11-af445bce8b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:30.752747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-e752408d-5aac-4ab4-8f11-af445bce8b49 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:30.754262 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-e752408d-5aac-4ab4-8f11-af445bce8b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:30.820544 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1188/2415] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:30.888964 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4804135-1914-4aa8-8388-e00b5c735eb2 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:30.889491 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4804135-1914-4aa8-8388-e00b5c735eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:30.889491 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4804135-1914-4aa8-8388-e00b5c735eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:30.910844 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-e4804135-1914-4aa8-8388-e00b5c735eb2 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:30.912093 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-e4804135-1914-4aa8-8388-e00b5c735eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:30.920364 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1228/2416] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:30.924427 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752408d-5aac-4ab4-8f11-af445bce8b49 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:30.934444 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752408d-5aac-4ab4-8f11-af445bce8b49 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:30.939126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a0099-472c-4686-bb29-7dab58c86d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:30.939741 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a0099-472c-4686-bb29-7dab58c86d37 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:30.940024 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a0099-472c-4686-bb29-7dab58c86d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:30.940305 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a0099-472c-4686-bb29-7dab58c86d37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:30.940975 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a0099-472c-4686-bb29-7dab58c86d37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:30.941281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a0099-472c-4686-bb29-7dab58c86d37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:30.942886 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a0099-472c-4686-bb29-7dab58c86d37 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:30.956365 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1189/2417] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:30.960443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4804135-1914-4aa8-8388-e00b5c735eb2 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:30.972559 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4804135-1914-4aa8-8388-e00b5c735eb2 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:30.982290 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7dc9b6-4b91-462d-91de-2b50a7523715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:30.982946 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7dc9b6-4b91-462d-91de-2b50a7523715 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:30.983267 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7dc9b6-4b91-462d-91de-2b50a7523715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:30.983548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7dc9b6-4b91-462d-91de-2b50a7523715 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:30.984245 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7dc9b6-4b91-462d-91de-2b50a7523715 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:30.987263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7dc9b6-4b91-462d-91de-2b50a7523715 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:31.023347 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1229/2418] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:15:30 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:15:31.024446 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a0099-472c-4686-bb29-7dab58c86d37 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:31.035313 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a0099-472c-4686-bb29-7dab58c86d37 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:31.129035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cdc732-a7ef-4a55-9eaf-784be987470b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:31.129984 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdc732-a7ef-4a55-9eaf-784be987470b admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:31.130366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdc732-a7ef-4a55-9eaf-784be987470b admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:31.130759 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdc732-a7ef-4a55-9eaf-784be987470b admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:31.131551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cdc732-a7ef-4a55-9eaf-784be987470b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:31.133567 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cdc732-a7ef-4a55-9eaf-784be987470b admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:31.157177 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1190/2419] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:15:31 2022] POST /identity/v3/users => generated 294 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:15:31.161697 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7dc9b6-4b91-462d-91de-2b50a7523715 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:31.172291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7dc9b6-4b91-462d-91de-2b50a7523715 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:31.177163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98c8e2-ebbb-4f43-81bc-61c8620d711e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:31.177761 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98c8e2-ebbb-4f43-81bc-61c8620d711e admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:31.178042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98c8e2-ebbb-4f43-81bc-61c8620d711e admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:31.178313 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98c8e2-ebbb-4f43-81bc-61c8620d711e admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:31.178952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98c8e2-ebbb-4f43-81bc-61c8620d711e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:31.180626 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98c8e2-ebbb-4f43-81bc-61c8620d711e admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:31.191297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1230/2420] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:15:31.195160 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cdc732-a7ef-4a55-9eaf-784be987470b admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:31.206785 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cdc732-a7ef-4a55-9eaf-784be987470b admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:31.211634 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ff3d1-1447-4c1d-9983-5126d39ed24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:31.212249 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ff3d1-1447-4c1d-9983-5126d39ed24a admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:31.212567 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ff3d1-1447-4c1d-9983-5126d39ed24a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:31.212788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ff3d1-1447-4c1d-9983-5126d39ed24a admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:31.213434 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ff3d1-1447-4c1d-9983-5126d39ed24a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:31.215035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ff3d1-1447-4c1d-9983-5126d39ed24a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:31.228378 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1191/2421] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:31.232914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98c8e2-ebbb-4f43-81bc-61c8620d711e admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:31.243350 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98c8e2-ebbb-4f43-81bc-61c8620d711e admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:31.248071 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23f320-bf78-4787-b6be-84d0f765a6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:31.248731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23f320-bf78-4787-b6be-84d0f765a6d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:31.249010 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23f320-bf78-4787-b6be-84d0f765a6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:31.249283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23f320-bf78-4787-b6be-84d0f765a6d0 admin admin] PATH_INFO: `/v3/projects/334d69920ad94108ad7abc8f612319a2/users/55c43d83751a4db18d44767674d35de9/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:31.266652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23f320-bf78-4787-b6be-84d0f765a6d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=334d69920ad94108ad7abc8f612319a2, user_id=55c43d83751a4db18d44767674d35de9, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:31.269969 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23f320-bf78-4787-b6be-84d0f765a6d0 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:31.283735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1231/2422] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:15:31 2022] PUT /identity/v3/projects/334d69920ad94108ad7abc8f612319a2/users/55c43d83751a4db18d44767674d35de9/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:15:31.294775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5362215-6582-4940-925f-4b6914bcd5a5 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:31.295091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5362215-6582-4940-925f-4b6914bcd5a5 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:31.295409 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5362215-6582-4940-925f-4b6914bcd5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:31.327854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-c5362215-6582-4940-925f-4b6914bcd5a5 None None] MFA Rules not processed for user `55c43d83751a4db18d44767674d35de9`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:31.329119 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c5362215-6582-4940-925f-4b6914bcd5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:31.386780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:31.417772 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1192/2423] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:31.422238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23f320-bf78-4787-b6be-84d0f765a6d0 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:31.438932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:31.509475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23f320-bf78-4787-b6be-84d0f765a6d0 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:31.514209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a99b0e-5916-4e9d-a7fc-8171fd119108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:31.515154 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a99b0e-5916-4e9d-a7fc-8171fd119108 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:31.515445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a99b0e-5916-4e9d-a7fc-8171fd119108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:31.515874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a99b0e-5916-4e9d-a7fc-8171fd119108 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:31.516626 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a99b0e-5916-4e9d-a7fc-8171fd119108 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:31.519595 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a99b0e-5916-4e9d-a7fc-8171fd119108 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:31.559113 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1232/2424] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:15:31 2022] POST /identity/v3/projects => generated 387 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:31.560726 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5362215-6582-4940-925f-4b6914bcd5a5 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:31.578405 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5362215-6582-4940-925f-4b6914bcd5a5 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:31.673337 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b11d2-1db9-406c-b83b-e9d91a184186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:31.673923 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b11d2-1db9-406c-b83b-e9d91a184186 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:31.674199 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b11d2-1db9-406c-b83b-e9d91a184186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:31.674468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b11d2-1db9-406c-b83b-e9d91a184186 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:31.675115 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b11d2-1db9-406c-b83b-e9d91a184186 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:31.677040 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b11d2-1db9-406c-b83b-e9d91a184186 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:31.701696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1193/2425] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:15:31 2022] POST /identity/v3/users => generated 300 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:31.706652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a99b0e-5916-4e9d-a7fc-8171fd119108 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:31.717948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a99b0e-5916-4e9d-a7fc-8171fd119108 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:31.722718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd33e5-d2db-49a6-b4fa-8aa3c75d72dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:31.723369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd33e5-d2db-49a6-b4fa-8aa3c75d72dd admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:31.723650 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd33e5-d2db-49a6-b4fa-8aa3c75d72dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:31.723922 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cd33e5-d2db-49a6-b4fa-8aa3c75d72dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:31.724668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd33e5-d2db-49a6-b4fa-8aa3c75d72dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:31.726236 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cd33e5-d2db-49a6-b4fa-8aa3c75d72dd admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:31.741770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1233/2426] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:31.743326 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b11d2-1db9-406c-b83b-e9d91a184186 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:31.754446 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b11d2-1db9-406c-b83b-e9d91a184186 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:31.759247 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c254bd5-ff9a-4895-a051-70a9cf6a4ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:31.759904 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c254bd5-ff9a-4895-a051-70a9cf6a4ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:31.760185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c254bd5-ff9a-4895-a051-70a9cf6a4ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:31.760458 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c254bd5-ff9a-4895-a051-70a9cf6a4ff4 admin admin] PATH_INFO: `/v3/projects/ca75fd24987a4b768ba6327bd9bfa55b/users/813d5b3957fc4b299a12a1a484997aa3/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:31.776377 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c254bd5-ff9a-4895-a051-70a9cf6a4ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca75fd24987a4b768ba6327bd9bfa55b, user_id=813d5b3957fc4b299a12a1a484997aa3, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:31.779130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c254bd5-ff9a-4895-a051-70a9cf6a4ff4 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:31.794244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1194/2427] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:15:31 2022] PUT /identity/v3/projects/ca75fd24987a4b768ba6327bd9bfa55b/users/813d5b3957fc4b299a12a1a484997aa3/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:31.798373 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd33e5-d2db-49a6-b4fa-8aa3c75d72dd admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:31.809360 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cd33e5-d2db-49a6-b4fa-8aa3c75d72dd admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:31.876560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabbe820-115a-411d-9a2e-c0261288fd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:31.877930 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbe820-115a-411d-9a2e-c0261288fd82 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:31.878296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbe820-115a-411d-9a2e-c0261288fd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:31.878719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabbe820-115a-411d-9a2e-c0261288fd82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:31.879449 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabbe820-115a-411d-9a2e-c0261288fd82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:31.881121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabbe820-115a-411d-9a2e-c0261288fd82 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:31.896596 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1234/2428] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:31.898012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c254bd5-ff9a-4895-a051-70a9cf6a4ff4 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:31.908852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c254bd5-ff9a-4895-a051-70a9cf6a4ff4 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:31.913654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf3ffc-e4c2-46ae-9e75-db1a7934afbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:31.914526 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf3ffc-e4c2-46ae-9e75-db1a7934afbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:31.914804 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf3ffc-e4c2-46ae-9e75-db1a7934afbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:31.915092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf3ffc-e4c2-46ae-9e75-db1a7934afbc admin admin] PATH_INFO: `/v3/projects/ca75fd24987a4b768ba6327bd9bfa55b/users/813d5b3957fc4b299a12a1a484997aa3/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:31.917090 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf3ffc-e4c2-46ae-9e75-db1a7934afbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca75fd24987a4b768ba6327bd9bfa55b, user_id=813d5b3957fc4b299a12a1a484997aa3, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:31.920572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf3ffc-e4c2-46ae-9e75-db1a7934afbc admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:31.932176 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1195/2429] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:15:31 2022] PUT /identity/v3/projects/ca75fd24987a4b768ba6327bd9bfa55b/users/813d5b3957fc4b299a12a1a484997aa3/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:31.942833 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de342796-eaa9-4838-899b-3fd350ea1e16 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:31.943233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de342796-eaa9-4838-899b-3fd350ea1e16 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:31.943524 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de342796-eaa9-4838-899b-3fd350ea1e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:31.974214 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-de342796-eaa9-4838-899b-3fd350ea1e16 None None] MFA Rules not processed for user `813d5b3957fc4b299a12a1a484997aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:31.975607 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-de342796-eaa9-4838-899b-3fd350ea1e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:32.053163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1235/2430] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:31 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:32.065162 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf3ffc-e4c2-46ae-9e75-db1a7934afbc admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:32.067501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-adbf3ffc-e4c2-46ae-9e75-db1a7934afbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:32.133253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf3ffc-e4c2-46ae-9e75-db1a7934afbc admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:32.138119 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d580e-69cb-42e9-85c2-184cf09d06dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:32.138927 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d580e-69cb-42e9-85c2-184cf09d06dd service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:32.139328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d580e-69cb-42e9-85c2-184cf09d06dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:32.139694 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d580e-69cb-42e9-85c2-184cf09d06dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:32.149647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d580e-69cb-42e9-85c2-184cf09d06dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:32.151193 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d580e-69cb-42e9-85c2-184cf09d06dd service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:32.162333 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1196/2431] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:15:32 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:15:32.318490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de342796-eaa9-4838-899b-3fd350ea1e16 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:32.329144 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:32.415370 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de342796-eaa9-4838-899b-3fd350ea1e16 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:32.420402 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b4788-6925-46d2-b1a7-e0cadb6ce179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:32.421393 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b4788-6925-46d2-b1a7-e0cadb6ce179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:32.421736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b4788-6925-46d2-b1a7-e0cadb6ce179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:32.422103 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b4788-6925-46d2-b1a7-e0cadb6ce179 admin admin] PATH_INFO: `/v3/users/55c43d83751a4db18d44767674d35de9` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:32.423836 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b4788-6925-46d2-b1a7-e0cadb6ce179 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55c43d83751a4db18d44767674d35de9)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:32.426212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b4788-6925-46d2-b1a7-e0cadb6ce179 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:32.484756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-438b4788-6925-46d2-b1a7-e0cadb6ce179 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55c43d83751a4db18d44767674d35de9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:32.492313 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-438b4788-6925-46d2-b1a7-e0cadb6ce179 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55c43d83751a4db18d44767674d35de9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:32.499368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-438b4788-6925-46d2-b1a7-e0cadb6ce179 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55c43d83751a4db18d44767674d35de9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:32.499983 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-438b4788-6925-46d2-b1a7-e0cadb6ce179 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55c43d83751a4db18d44767674d35de9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:32.500610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-438b4788-6925-46d2-b1a7-e0cadb6ce179 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55c43d83751a4db18d44767674d35de9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:32.525407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1236/2432] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:15:32 2022] DELETE /identity/v3/users/55c43d83751a4db18d44767674d35de9 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:32.530000 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cefc0-708d-4d45-a8e0-1640b0949891 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:32.530302 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cefc0-708d-4d45-a8e0-1640b0949891 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:32.530601 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8cefc0-708d-4d45-a8e0-1640b0949891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:32.552138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-1a8cefc0-708d-4d45-a8e0-1640b0949891 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:32.553414 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1a8cefc0-708d-4d45-a8e0-1640b0949891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:32.597156 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:32.627677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1197/2433] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:15:32.638483 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b4788-6925-46d2-b1a7-e0cadb6ce179 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:32.640036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-438b4788-6925-46d2-b1a7-e0cadb6ce179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:32.713677 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b4788-6925-46d2-b1a7-e0cadb6ce179 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:32.717421 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d61fac-7b9f-4dc6-a8ad-6198d40d6e27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:32.718026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d61fac-7b9f-4dc6-a8ad-6198d40d6e27 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:32.718427 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d61fac-7b9f-4dc6-a8ad-6198d40d6e27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:32.718569 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d61fac-7b9f-4dc6-a8ad-6198d40d6e27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:32.734148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8cefc0-708d-4d45-a8e0-1640b0949891 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:32.735797 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1a8cefc0-708d-4d45-a8e0-1640b0949891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:32.735923 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d61fac-7b9f-4dc6-a8ad-6198d40d6e27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:32.737562 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d61fac-7b9f-4dc6-a8ad-6198d40d6e27 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:32.748295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1237/2434] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:15:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:15:32.749253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8cefc0-708d-4d45-a8e0-1640b0949891 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:32.753421 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087e748-1a17-4826-b7af-b24fbe6a5b3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:32.754036 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087e748-1a17-4826-b7af-b24fbe6a5b3a service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:32.754938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087e748-1a17-4826-b7af-b24fbe6a5b3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:32.754938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087e748-1a17-4826-b7af-b24fbe6a5b3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:32.756644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c087e748-1a17-4826-b7af-b24fbe6a5b3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:32.769958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087e748-1a17-4826-b7af-b24fbe6a5b3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:32.771503 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087e748-1a17-4826-b7af-b24fbe6a5b3a service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:32.789913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1198/2435] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:15:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:15:33.124372 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d61fac-7b9f-4dc6-a8ad-6198d40d6e27 service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:33.129097 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-43d61fac-7b9f-4dc6-a8ad-6198d40d6e27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:33.155377 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d61fac-7b9f-4dc6-a8ad-6198d40d6e27 service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:33.170590 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa46faa-0ccc-44ae-bc1a-2bc4cef836c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:33.171272 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa46faa-0ccc-44ae-bc1a-2bc4cef836c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:33.171631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa46faa-0ccc-44ae-bc1a-2bc4cef836c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:33.171888 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa46faa-0ccc-44ae-bc1a-2bc4cef836c4 admin admin] PATH_INFO: `/v3/projects/334d69920ad94108ad7abc8f612319a2` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:33.173025 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa46faa-0ccc-44ae-bc1a-2bc4cef836c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=334d69920ad94108ad7abc8f612319a2)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:33.176571 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa46faa-0ccc-44ae-bc1a-2bc4cef836c4 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:33.245242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:33.274373 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5fa46faa-0ccc-44ae-bc1a-2bc4cef836c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '334d69920ad94108ad7abc8f612319a2', 'request_id': 'req-5fa46faa-0ccc-44ae-bc1a-2bc4cef836c4'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:33.289030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5fa46faa-0ccc-44ae-bc1a-2bc4cef836c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '334d69920ad94108ad7abc8f612319a2', 'request_id': 'req-5fa46faa-0ccc-44ae-bc1a-2bc4cef836c4'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:33.295498 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5fa46faa-0ccc-44ae-bc1a-2bc4cef836c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '334d69920ad94108ad7abc8f612319a2', 'request_id': 'req-5fa46faa-0ccc-44ae-bc1a-2bc4cef836c4'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:33.309633 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5fa46faa-0ccc-44ae-bc1a-2bc4cef836c4 admin admin] The token cache is being invalidate because project 334d69920ad94108ad7abc8f612319a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:15:33.310001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5fa46faa-0ccc-44ae-bc1a-2bc4cef836c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:33.310673 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5fa46faa-0ccc-44ae-bc1a-2bc4cef836c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:33.312510 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1238/2436] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:15:33 2022] DELETE /identity/v3/projects/334d69920ad94108ad7abc8f612319a2 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:33.317080 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087e748-1a17-4826-b7af-b24fbe6a5b3a service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:33.332805 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087e748-1a17-4826-b7af-b24fbe6a5b3a service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:33.417752 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6adfd-0047-4bad-a54d-b956c61a9c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:33.419106 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6adfd-0047-4bad-a54d-b956c61a9c8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:33.420165 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6adfd-0047-4bad-a54d-b956c61a9c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:33.420563 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6adfd-0047-4bad-a54d-b956c61a9c8a admin admin] PATH_INFO: `/v3/users/813d5b3957fc4b299a12a1a484997aa3` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:33.421901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6adfd-0047-4bad-a54d-b956c61a9c8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=813d5b3957fc4b299a12a1a484997aa3)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:33.425244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6adfd-0047-4bad-a54d-b956c61a9c8a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:33.499508 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-35b6adfd-0047-4bad-a54d-b956c61a9c8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '813d5b3957fc4b299a12a1a484997aa3'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:33.500595 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-35b6adfd-0047-4bad-a54d-b956c61a9c8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '813d5b3957fc4b299a12a1a484997aa3'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:33.519710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-35b6adfd-0047-4bad-a54d-b956c61a9c8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '813d5b3957fc4b299a12a1a484997aa3'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:33.532383 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-35b6adfd-0047-4bad-a54d-b956c61a9c8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '813d5b3957fc4b299a12a1a484997aa3'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:33.533800 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-35b6adfd-0047-4bad-a54d-b956c61a9c8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '813d5b3957fc4b299a12a1a484997aa3'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:33.548369 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1199/2437] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:15:33 2022] DELETE /identity/v3/users/813d5b3957fc4b299a12a1a484997aa3 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:33.923635 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa46faa-0ccc-44ae-bc1a-2bc4cef836c4 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:33.929879 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5fa46faa-0ccc-44ae-bc1a-2bc4cef836c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:33.954085 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa46faa-0ccc-44ae-bc1a-2bc4cef836c4 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:34.068550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9d62c5-1816-4d21-886a-41509b1a32f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:34.069198 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d62c5-1816-4d21-886a-41509b1a32f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:34.069509 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d62c5-1816-4d21-886a-41509b1a32f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:34.069753 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9d62c5-1816-4d21-886a-41509b1a32f2 admin admin] PATH_INFO: `/v3/projects/ca75fd24987a4b768ba6327bd9bfa55b` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:34.071176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d62c5-1816-4d21-886a-41509b1a32f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca75fd24987a4b768ba6327bd9bfa55b)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:34.073003 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9d62c5-1816-4d21-886a-41509b1a32f2 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:34.175906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-3c9d62c5-1816-4d21-886a-41509b1a32f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca75fd24987a4b768ba6327bd9bfa55b', 'request_id': 'req-3c9d62c5-1816-4d21-886a-41509b1a32f2'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:34.191708 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-3c9d62c5-1816-4d21-886a-41509b1a32f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca75fd24987a4b768ba6327bd9bfa55b', 'request_id': 'req-3c9d62c5-1816-4d21-886a-41509b1a32f2'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:34.200246 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-3c9d62c5-1816-4d21-886a-41509b1a32f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca75fd24987a4b768ba6327bd9bfa55b', 'request_id': 'req-3c9d62c5-1816-4d21-886a-41509b1a32f2'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:34.215942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-3c9d62c5-1816-4d21-886a-41509b1a32f2 admin admin] The token cache is being invalidate because project ca75fd24987a4b768ba6327bd9bfa55b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:15:34.220445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-3c9d62c5-1816-4d21-886a-41509b1a32f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:34.224885 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-3c9d62c5-1816-4d21-886a-41509b1a32f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:34.227132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1239/2438] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:15:33 2022] DELETE /identity/v3/projects/ca75fd24987a4b768ba6327bd9bfa55b => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:15:34.232092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebe9b9-4292-4aa0-9c9e-d131e58eb04b None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:34.232771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebe9b9-4292-4aa0-9c9e-d131e58eb04b None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:34.232771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebe9b9-4292-4aa0-9c9e-d131e58eb04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:34.262242 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-4eebe9b9-4292-4aa0-9c9e-d131e58eb04b None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:34.265699 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-4eebe9b9-4292-4aa0-9c9e-d131e58eb04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:34.345051 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:34.386954 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1200/2439] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:15:34.388658 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361f72c-fb6b-4a57-bc32-1c2cdc9325b7 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:34.388944 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361f72c-fb6b-4a57-bc32-1c2cdc9325b7 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:34.389214 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6361f72c-fb6b-4a57-bc32-1c2cdc9325b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:34.416498 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-6361f72c-fb6b-4a57-bc32-1c2cdc9325b7 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:34.417714 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6361f72c-fb6b-4a57-bc32-1c2cdc9325b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:34.427120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1240/2440] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:34.430131 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebe9b9-4292-4aa0-9c9e-d131e58eb04b None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:34.441323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebe9b9-4292-4aa0-9c9e-d131e58eb04b None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:34.450514 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263d737-d416-454b-b673-00a5dc0d9798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:34.451312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263d737-d416-454b-b673-00a5dc0d9798 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:34.451705 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263d737-d416-454b-b673-00a5dc0d9798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:34.452735 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4263d737-d416-454b-b673-00a5dc0d9798 admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:34.453499 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263d737-d416-454b-b673-00a5dc0d9798 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:34.453900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263d737-d416-454b-b673-00a5dc0d9798 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:34.456705 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4263d737-d416-454b-b673-00a5dc0d9798 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:34.474463 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361f72c-fb6b-4a57-bc32-1c2cdc9325b7 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:34.475178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1201/2441] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:34.485046 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6361f72c-fb6b-4a57-bc32-1c2cdc9325b7 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:34.489839 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31fcb15-31b7-48bf-a03c-7acc7b555692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:34.490441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31fcb15-31b7-48bf-a03c-7acc7b555692 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:34.490718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31fcb15-31b7-48bf-a03c-7acc7b555692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:34.490986 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31fcb15-31b7-48bf-a03c-7acc7b555692 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:34.491654 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31fcb15-31b7-48bf-a03c-7acc7b555692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:34.493552 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31fcb15-31b7-48bf-a03c-7acc7b555692 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:34.535960 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1241/2442] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:15:34 2022] POST /identity/v3/projects => generated 405 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:15:34.536820 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263d737-d416-454b-b673-00a5dc0d9798 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:34.550189 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4263d737-d416-454b-b673-00a5dc0d9798 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:34.612649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5e980-aef3-4910-b944-eaeee0a9892c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:34.613283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5e980-aef3-4910-b944-eaeee0a9892c admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:34.613565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5e980-aef3-4910-b944-eaeee0a9892c admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:34.613844 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5e980-aef3-4910-b944-eaeee0a9892c admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:34.614477 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5e980-aef3-4910-b944-eaeee0a9892c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:34.616551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5e980-aef3-4910-b944-eaeee0a9892c admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:34.644293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1202/2443] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:15:34 2022] POST /identity/v3/users => generated 303 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:15:34.648766 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31fcb15-31b7-48bf-a03c-7acc7b555692 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:34.661182 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31fcb15-31b7-48bf-a03c-7acc7b555692 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:34.687731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae7d50-4f06-4456-af38-4d408a8b2637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:34.688735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae7d50-4f06-4456-af38-4d408a8b2637 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:34.689292 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae7d50-4f06-4456-af38-4d408a8b2637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:34.690671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae7d50-4f06-4456-af38-4d408a8b2637 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:34.691603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae7d50-4f06-4456-af38-4d408a8b2637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:34.693331 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ae7d50-4f06-4456-af38-4d408a8b2637 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:34.717338 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1242/2444] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:34.717997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5e980-aef3-4910-b944-eaeee0a9892c admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:34.744922 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5e980-aef3-4910-b944-eaeee0a9892c admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:34.783316 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ca258-6fe2-4fa1-ae57-c7ccf8d1e347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:34.784034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ca258-6fe2-4fa1-ae57-c7ccf8d1e347 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:34.784413 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ca258-6fe2-4fa1-ae57-c7ccf8d1e347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:34.784753 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ca258-6fe2-4fa1-ae57-c7ccf8d1e347 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:34.787322 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ca258-6fe2-4fa1-ae57-c7ccf8d1e347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:34.790622 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ca258-6fe2-4fa1-ae57-c7ccf8d1e347 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:34.813573 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae7d50-4f06-4456-af38-4d408a8b2637 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:34.815098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1203/2445] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:15:34.826088 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae7d50-4f06-4456-af38-4d408a8b2637 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:34.828478 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:34.839501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded8574-497e-441b-853e-e34c63a33c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:34.841187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded8574-497e-441b-853e-e34c63a33c68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:34.842184 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded8574-497e-441b-853e-e34c63a33c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:34.843262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded8574-497e-441b-853e-e34c63a33c68 admin admin] PATH_INFO: `/v3/projects/46068c98896a4b7e8e84a49488b6b6b6/users/15155e8e1767492c88d540cb8d49f19f/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:34.879715 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded8574-497e-441b-853e-e34c63a33c68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46068c98896a4b7e8e84a49488b6b6b6, user_id=15155e8e1767492c88d540cb8d49f19f, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:34.882495 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded8574-497e-441b-853e-e34c63a33c68 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:34.907293 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1243/2446] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:15:34 2022] PUT /identity/v3/projects/46068c98896a4b7e8e84a49488b6b6b6/users/15155e8e1767492c88d540cb8d49f19f/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:34.918307 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d59217-68f9-40a6-bb9e-8817dc9cd8f1 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:34.918719 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d59217-68f9-40a6-bb9e-8817dc9cd8f1 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:34.919100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d59217-68f9-40a6-bb9e-8817dc9cd8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:35.000559 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-39d59217-68f9-40a6-bb9e-8817dc9cd8f1 None None] MFA Rules not processed for user `15155e8e1767492c88d540cb8d49f19f`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:35.007255 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-39d59217-68f9-40a6-bb9e-8817dc9cd8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:35.170406 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1204/2447] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:34 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:35.175321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded8574-497e-441b-853e-e34c63a33c68 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:35.328804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded8574-497e-441b-853e-e34c63a33c68 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:35.334899 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169530d-e4a7-47af-aa53-645b089fa2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:35.335615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169530d-e4a7-47af-aa53-645b089fa2d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:35.335963 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169530d-e4a7-47af-aa53-645b089fa2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:35.336295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169530d-e4a7-47af-aa53-645b089fa2d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:35.336941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169530d-e4a7-47af-aa53-645b089fa2d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:35.345835 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4169530d-e4a7-47af-aa53-645b089fa2d8 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:35.402502 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1244/2448] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:15:35 2022] POST /identity/v3/projects => generated 409 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:35.407030 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d59217-68f9-40a6-bb9e-8817dc9cd8f1 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:35.423142 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d59217-68f9-40a6-bb9e-8817dc9cd8f1 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:35.521427 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:35.560271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5061ad51-cf77-40e9-8be8-41d20da1b0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:35.561377 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061ad51-cf77-40e9-8be8-41d20da1b0f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:35.561871 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061ad51-cf77-40e9-8be8-41d20da1b0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:35.562328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5061ad51-cf77-40e9-8be8-41d20da1b0f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:35.563375 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5061ad51-cf77-40e9-8be8-41d20da1b0f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:35.566158 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5061ad51-cf77-40e9-8be8-41d20da1b0f8 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:35.613860 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1205/2449] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:15:35 2022] POST /identity/v3/users => generated 311 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:35.615204 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169530d-e4a7-47af-aa53-645b089fa2d8 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:35.635663 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169530d-e4a7-47af-aa53-645b089fa2d8 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:35.641222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65dd7db-18ed-4e3a-9453-82fd12b71068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:35.642107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65dd7db-18ed-4e3a-9453-82fd12b71068 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:35.642386 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65dd7db-18ed-4e3a-9453-82fd12b71068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:35.643233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65dd7db-18ed-4e3a-9453-82fd12b71068 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:35.644299 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65dd7db-18ed-4e3a-9453-82fd12b71068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:35.646348 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65dd7db-18ed-4e3a-9453-82fd12b71068 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:35.658677 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1245/2450] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:35.662786 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5061ad51-cf77-40e9-8be8-41d20da1b0f8 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:35.673085 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5061ad51-cf77-40e9-8be8-41d20da1b0f8 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:35.678358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d39d5b-5c0c-4cb2-9989-9ae3d1611972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:35.679030 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d39d5b-5c0c-4cb2-9989-9ae3d1611972 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:35.679336 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d39d5b-5c0c-4cb2-9989-9ae3d1611972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:35.679608 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d39d5b-5c0c-4cb2-9989-9ae3d1611972 admin admin] PATH_INFO: `/v3/projects/250c3b68567644ddbecbb33740efe672/users/63a2c63f1bd24caca2aa70aa28219474/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:35.704244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d39d5b-5c0c-4cb2-9989-9ae3d1611972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=250c3b68567644ddbecbb33740efe672, user_id=63a2c63f1bd24caca2aa70aa28219474, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:35.708343 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d39d5b-5c0c-4cb2-9989-9ae3d1611972 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:35.725533 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1206/2451] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:15:35 2022] PUT /identity/v3/projects/250c3b68567644ddbecbb33740efe672/users/63a2c63f1bd24caca2aa70aa28219474/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:35.729868 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65dd7db-18ed-4e3a-9453-82fd12b71068 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:35.740857 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65dd7db-18ed-4e3a-9453-82fd12b71068 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:35.839015 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29ec81-fc80-4c67-b593-d1566b82190b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:35.840276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29ec81-fc80-4c67-b593-d1566b82190b admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:35.840752 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29ec81-fc80-4c67-b593-d1566b82190b admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:35.841224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29ec81-fc80-4c67-b593-d1566b82190b admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:35.842662 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29ec81-fc80-4c67-b593-d1566b82190b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:35.845065 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29ec81-fc80-4c67-b593-d1566b82190b admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:35.862073 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1246/2452] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:35.862989 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d39d5b-5c0c-4cb2-9989-9ae3d1611972 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:35.874500 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d39d5b-5c0c-4cb2-9989-9ae3d1611972 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:35.880935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b2d5e-cd20-4d31-988d-61b052b2c870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:35.881603 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b2d5e-cd20-4d31-988d-61b052b2c870 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:35.881879 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b2d5e-cd20-4d31-988d-61b052b2c870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:35.882293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b2d5e-cd20-4d31-988d-61b052b2c870 admin admin] PATH_INFO: `/v3/projects/250c3b68567644ddbecbb33740efe672/users/63a2c63f1bd24caca2aa70aa28219474/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:35.884246 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b2d5e-cd20-4d31-988d-61b052b2c870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=250c3b68567644ddbecbb33740efe672, user_id=63a2c63f1bd24caca2aa70aa28219474, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:35.887092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b2d5e-cd20-4d31-988d-61b052b2c870 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:35.907233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1207/2453] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:15:35 2022] PUT /identity/v3/projects/250c3b68567644ddbecbb33740efe672/users/63a2c63f1bd24caca2aa70aa28219474/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:35.915795 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d4523-a773-49b0-8d69-7b3ae002412e None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:35.916092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d4523-a773-49b0-8d69-7b3ae002412e None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:35.916338 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d4523-a773-49b0-8d69-7b3ae002412e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:35.952733 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-0e5d4523-a773-49b0-8d69-7b3ae002412e None None] MFA Rules not processed for user `63a2c63f1bd24caca2aa70aa28219474`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:35.954595 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0e5d4523-a773-49b0-8d69-7b3ae002412e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:36.010224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:36.062316 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1247/2454] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:35 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:36.074780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b2d5e-cd20-4d31-988d-61b052b2c870 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:36.076880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-330b2d5e-cd20-4d31-988d-61b052b2c870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:36.158355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b2d5e-cd20-4d31-988d-61b052b2c870 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:36.170368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534605c-87be-47df-aac8-c8216207af3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:36.171775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534605c-87be-47df-aac8-c8216207af3d service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:36.172284 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534605c-87be-47df-aac8-c8216207af3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:36.172796 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d534605c-87be-47df-aac8-c8216207af3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:36.198469 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d534605c-87be-47df-aac8-c8216207af3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:36.201083 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d534605c-87be-47df-aac8-c8216207af3d service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:36.217032 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1208/2455] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:15:36 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:15:36.454679 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d4523-a773-49b0-8d69-7b3ae002412e None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:36.482004 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d4523-a773-49b0-8d69-7b3ae002412e None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:36.487573 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754d496-2b87-4da0-be92-2553a33aa53f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:36.488395 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754d496-2b87-4da0-be92-2553a33aa53f service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:36.488839 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754d496-2b87-4da0-be92-2553a33aa53f service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:36.489265 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754d496-2b87-4da0-be92-2553a33aa53f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:36.576261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754d496-2b87-4da0-be92-2553a33aa53f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:36.578108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754d496-2b87-4da0-be92-2553a33aa53f service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:36.593545 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1248/2456] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:15:36 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:15:37.167718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534605c-87be-47df-aac8-c8216207af3d service cinder] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:37.169139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-d534605c-87be-47df-aac8-c8216207af3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:37.240422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d534605c-87be-47df-aac8-c8216207af3d service cinder] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:37.245126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f125ba0e-5447-41d9-af7d-b9c5ab858453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:37.245763 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125ba0e-5447-41d9-af7d-b9c5ab858453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:37.246049 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125ba0e-5447-41d9-af7d-b9c5ab858453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:37.246324 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f125ba0e-5447-41d9-af7d-b9c5ab858453 admin admin] PATH_INFO: `/v3/users/4210f948dc7e4a5bac61d10eb6a62eee` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:37.247480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f125ba0e-5447-41d9-af7d-b9c5ab858453 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4210f948dc7e4a5bac61d10eb6a62eee)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:37.249421 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f125ba0e-5447-41d9-af7d-b9c5ab858453 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:37.289150 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:37.295726 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f125ba0e-5447-41d9-af7d-b9c5ab858453 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4210f948dc7e4a5bac61d10eb6a62eee'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:37.296644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f125ba0e-5447-41d9-af7d-b9c5ab858453 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4210f948dc7e4a5bac61d10eb6a62eee'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:37.309863 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f125ba0e-5447-41d9-af7d-b9c5ab858453 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4210f948dc7e4a5bac61d10eb6a62eee'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:37.317325 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f125ba0e-5447-41d9-af7d-b9c5ab858453 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4210f948dc7e4a5bac61d10eb6a62eee'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:37.318010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f125ba0e-5447-41d9-af7d-b9c5ab858453 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4210f948dc7e4a5bac61d10eb6a62eee'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:37.336687 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1209/2457] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:15:37 2022] DELETE /identity/v3/users/4210f948dc7e4a5bac61d10eb6a62eee => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:37.509126 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754d496-2b87-4da0-be92-2553a33aa53f service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:37.520707 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754d496-2b87-4da0-be92-2553a33aa53f service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:37.583207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57c977-49a5-4f17-ae76-dc1e07f14332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:37.583840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57c977-49a5-4f17-ae76-dc1e07f14332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:37.584167 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57c977-49a5-4f17-ae76-dc1e07f14332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:37.584438 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57c977-49a5-4f17-ae76-dc1e07f14332 admin admin] PATH_INFO: `/v3/projects/2d6a6fe5db0a452a83dde4aac26b0161` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:37.585551 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57c977-49a5-4f17-ae76-dc1e07f14332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d6a6fe5db0a452a83dde4aac26b0161)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:37.587517 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57c977-49a5-4f17-ae76-dc1e07f14332 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:37.633653 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-1f57c977-49a5-4f17-ae76-dc1e07f14332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d6a6fe5db0a452a83dde4aac26b0161', 'request_id': 'req-1f57c977-49a5-4f17-ae76-dc1e07f14332'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:37.641146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-1f57c977-49a5-4f17-ae76-dc1e07f14332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d6a6fe5db0a452a83dde4aac26b0161', 'request_id': 'req-1f57c977-49a5-4f17-ae76-dc1e07f14332'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:37.647344 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-1f57c977-49a5-4f17-ae76-dc1e07f14332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d6a6fe5db0a452a83dde4aac26b0161', 'request_id': 'req-1f57c977-49a5-4f17-ae76-dc1e07f14332'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:37.658926 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-1f57c977-49a5-4f17-ae76-dc1e07f14332 admin admin] The token cache is being invalidate because project 2d6a6fe5db0a452a83dde4aac26b0161 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:15:37.659253 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-1f57c977-49a5-4f17-ae76-dc1e07f14332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:37.659899 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-1f57c977-49a5-4f17-ae76-dc1e07f14332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:37.661411 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1249/2458] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:15:37 2022] DELETE /identity/v3/projects/2d6a6fe5db0a452a83dde4aac26b0161 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:37.666310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f437b-3f24-42ea-83be-64c929d677f5 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:37.666594 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f437b-3f24-42ea-83be-64c929d677f5 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:37.666894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f437b-3f24-42ea-83be-64c929d677f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:37.686855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-b22f437b-3f24-42ea-83be-64c929d677f5 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:37.688121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b22f437b-3f24-42ea-83be-64c929d677f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:37.750870 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1210/2459] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:37.759651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd68a25-f351-40b7-bc75-320c416ac452 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:37.759951 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd68a25-f351-40b7-bc75-320c416ac452 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:37.760223 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd68a25-f351-40b7-bc75-320c416ac452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:37.768746 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:37.780621 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-8fd68a25-f351-40b7-bc75-320c416ac452 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:37.781799 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8fd68a25-f351-40b7-bc75-320c416ac452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:37.789544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1250/2460] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:37.793863 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f437b-3f24-42ea-83be-64c929d677f5 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:37.803872 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f437b-3f24-42ea-83be-64c929d677f5 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:37.808422 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f1140-e3e5-4f1e-870a-362ad027dd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:37.809008 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f1140-e3e5-4f1e-870a-362ad027dd15 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:37.809290 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f1140-e3e5-4f1e-870a-362ad027dd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:37.809558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f1140-e3e5-4f1e-870a-362ad027dd15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:37.810199 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f1140-e3e5-4f1e-870a-362ad027dd15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:37.810496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f1140-e3e5-4f1e-870a-362ad027dd15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:37.812184 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f1140-e3e5-4f1e-870a-362ad027dd15 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:37.825470 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1211/2461] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:37.830092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd68a25-f351-40b7-bc75-320c416ac452 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:37.842710 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd68a25-f351-40b7-bc75-320c416ac452 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:37.847393 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92cb77-f6a6-4fd5-9821-26dbc05863a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:37.847986 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92cb77-f6a6-4fd5-9821-26dbc05863a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:37.848262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92cb77-f6a6-4fd5-9821-26dbc05863a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:37.848529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92cb77-f6a6-4fd5-9821-26dbc05863a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:37.849153 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92cb77-f6a6-4fd5-9821-26dbc05863a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:37.851034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92cb77-f6a6-4fd5-9821-26dbc05863a8 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:37.879306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1251/2462] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:15:37 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:37.883309 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f1140-e3e5-4f1e-870a-362ad027dd15 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:37.893451 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f1140-e3e5-4f1e-870a-362ad027dd15 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:37.948964 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a10724-aaec-4efc-b997-7ef6f63bd00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:37.949619 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a10724-aaec-4efc-b997-7ef6f63bd00f admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:37.949913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a10724-aaec-4efc-b997-7ef6f63bd00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:37.950187 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a10724-aaec-4efc-b997-7ef6f63bd00f admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:37.950809 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a10724-aaec-4efc-b997-7ef6f63bd00f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:37.952754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a10724-aaec-4efc-b997-7ef6f63bd00f admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:37.977742 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1212/2463] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:15:37 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:37.978896 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92cb77-f6a6-4fd5-9821-26dbc05863a8 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:37.989628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92cb77-f6a6-4fd5-9821-26dbc05863a8 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:37.994484 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34c358-58cb-4b1f-b9e1-4dda98c94067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:37.995111 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34c358-58cb-4b1f-b9e1-4dda98c94067 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:37.995397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34c358-58cb-4b1f-b9e1-4dda98c94067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:37.995669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf34c358-58cb-4b1f-b9e1-4dda98c94067 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:37.996309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34c358-58cb-4b1f-b9e1-4dda98c94067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:37.997890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf34c358-58cb-4b1f-b9e1-4dda98c94067 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:38.008736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1252/2464] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:38.012831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a10724-aaec-4efc-b997-7ef6f63bd00f admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:38.026527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a10724-aaec-4efc-b997-7ef6f63bd00f admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:38.031547 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6040d4-5af1-4462-b5e7-c93d527a664c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:38.032203 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6040d4-5af1-4462-b5e7-c93d527a664c admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:38.032483 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6040d4-5af1-4462-b5e7-c93d527a664c admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:38.032802 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6040d4-5af1-4462-b5e7-c93d527a664c admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:38.033443 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6040d4-5af1-4462-b5e7-c93d527a664c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:38.035022 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6040d4-5af1-4462-b5e7-c93d527a664c admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:38.045738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1213/2465] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:38.050244 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34c358-58cb-4b1f-b9e1-4dda98c94067 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:38.060058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf34c358-58cb-4b1f-b9e1-4dda98c94067 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:38.065108 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6fd8e-81d8-48aa-8101-622dc0e1362e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:38.065743 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6fd8e-81d8-48aa-8101-622dc0e1362e admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:38.066023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6fd8e-81d8-48aa-8101-622dc0e1362e admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:38.066295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6fd8e-81d8-48aa-8101-622dc0e1362e admin admin] PATH_INFO: `/v3/projects/e6d44d0c4b87445f95258b954e5f576d/users/01a33cc6ec4c4c0589e7b556c90bc561/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:38.080995 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6fd8e-81d8-48aa-8101-622dc0e1362e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d44d0c4b87445f95258b954e5f576d, user_id=01a33cc6ec4c4c0589e7b556c90bc561, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:38.083938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6fd8e-81d8-48aa-8101-622dc0e1362e admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:38.098413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1253/2466] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:15:38 2022] PUT /identity/v3/projects/e6d44d0c4b87445f95258b954e5f576d/users/01a33cc6ec4c4c0589e7b556c90bc561/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:38.110233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232ce8c-675c-43d5-a41a-3cb388d0bf23 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:38.110597 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232ce8c-675c-43d5-a41a-3cb388d0bf23 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:38.110958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232ce8c-675c-43d5-a41a-3cb388d0bf23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:38.139547 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-c232ce8c-675c-43d5-a41a-3cb388d0bf23 None None] MFA Rules not processed for user `01a33cc6ec4c4c0589e7b556c90bc561`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:38.140972 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c232ce8c-675c-43d5-a41a-3cb388d0bf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:38.151377 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:38.209641 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1214/2467] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:38.214909 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6fd8e-81d8-48aa-8101-622dc0e1362e admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:38.285832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6fd8e-81d8-48aa-8101-622dc0e1362e admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:38.293190 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43760ee9-9ee3-4dcb-9313-b1b7353a5596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:38.293824 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43760ee9-9ee3-4dcb-9313-b1b7353a5596 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:38.294174 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43760ee9-9ee3-4dcb-9313-b1b7353a5596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:38.294604 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43760ee9-9ee3-4dcb-9313-b1b7353a5596 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:38.295590 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43760ee9-9ee3-4dcb-9313-b1b7353a5596 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:38.297907 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43760ee9-9ee3-4dcb-9313-b1b7353a5596 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:38.333015 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1254/2468] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:15:38 2022] POST /identity/v3/projects => generated 381 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:38.337679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232ce8c-675c-43d5-a41a-3cb388d0bf23 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:38.355781 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232ce8c-675c-43d5-a41a-3cb388d0bf23 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:38.411535 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef0581-02e8-4153-9780-f61a82088755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:38.412094 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef0581-02e8-4153-9780-f61a82088755 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:38.412373 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef0581-02e8-4153-9780-f61a82088755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:38.412645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ef0581-02e8-4153-9780-f61a82088755 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:38.413266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef0581-02e8-4153-9780-f61a82088755 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:38.415200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ef0581-02e8-4153-9780-f61a82088755 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:38.434677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1215/2469] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:15:38 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:38.438677 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43760ee9-9ee3-4dcb-9313-b1b7353a5596 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:38.449478 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43760ee9-9ee3-4dcb-9313-b1b7353a5596 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:38.455489 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f42aff-4452-4a69-8e08-5b1782bd4307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:38.456165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f42aff-4452-4a69-8e08-5b1782bd4307 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:38.456720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f42aff-4452-4a69-8e08-5b1782bd4307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:38.456993 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f42aff-4452-4a69-8e08-5b1782bd4307 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:38.457631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f42aff-4452-4a69-8e08-5b1782bd4307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:38.459248 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f42aff-4452-4a69-8e08-5b1782bd4307 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:38.472334 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1255/2470] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:38.476598 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef0581-02e8-4153-9780-f61a82088755 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:38.486470 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ef0581-02e8-4153-9780-f61a82088755 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:38.491118 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb41fa-1b09-4d1c-9898-b137354e2649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:38.491762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb41fa-1b09-4d1c-9898-b137354e2649 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:38.492041 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb41fa-1b09-4d1c-9898-b137354e2649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:38.492353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb41fa-1b09-4d1c-9898-b137354e2649 admin admin] PATH_INFO: `/v3/projects/8c43fb7a17314603b6addef503d297a6/users/5a614e9e13cb403f8f0b4cb199708089/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:38.506563 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb41fa-1b09-4d1c-9898-b137354e2649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c43fb7a17314603b6addef503d297a6, user_id=5a614e9e13cb403f8f0b4cb199708089, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:38.509251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb41fa-1b09-4d1c-9898-b137354e2649 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:38.522333 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1216/2471] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:15:38 2022] PUT /identity/v3/projects/8c43fb7a17314603b6addef503d297a6/users/5a614e9e13cb403f8f0b4cb199708089/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:38.526146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f42aff-4452-4a69-8e08-5b1782bd4307 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:38.536523 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f42aff-4452-4a69-8e08-5b1782bd4307 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:38.592012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852d276-e649-4021-86fe-b0780b386b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:38.592608 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852d276-e649-4021-86fe-b0780b386b8a admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:38.592887 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852d276-e649-4021-86fe-b0780b386b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:38.593159 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852d276-e649-4021-86fe-b0780b386b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:38.593789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852d276-e649-4021-86fe-b0780b386b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:38.595425 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852d276-e649-4021-86fe-b0780b386b8a admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:38.605705 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1256/2472] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:38.609965 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb41fa-1b09-4d1c-9898-b137354e2649 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:38.620270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb41fa-1b09-4d1c-9898-b137354e2649 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:38.624855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433f36b-2a9a-4cf9-a2ef-d3003f8829f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:38.625492 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433f36b-2a9a-4cf9-a2ef-d3003f8829f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:38.625778 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433f36b-2a9a-4cf9-a2ef-d3003f8829f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:38.626049 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433f36b-2a9a-4cf9-a2ef-d3003f8829f4 admin admin] PATH_INFO: `/v3/projects/8c43fb7a17314603b6addef503d297a6/users/5a614e9e13cb403f8f0b4cb199708089/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:38.627999 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1433f36b-2a9a-4cf9-a2ef-d3003f8829f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c43fb7a17314603b6addef503d297a6, user_id=5a614e9e13cb403f8f0b4cb199708089, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:38.630624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1433f36b-2a9a-4cf9-a2ef-d3003f8829f4 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:38.640672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1217/2473] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:15:38 2022] PUT /identity/v3/projects/8c43fb7a17314603b6addef503d297a6/users/5a614e9e13cb403f8f0b4cb199708089/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:38.651239 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78b5e2-2b27-49c9-983a-ae627451e2ee None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:38.651532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78b5e2-2b27-49c9-983a-ae627451e2ee None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:38.651806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78b5e2-2b27-49c9-983a-ae627451e2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:38.672186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:38.680242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-9b78b5e2-2b27-49c9-983a-ae627451e2ee None None] MFA Rules not processed for user `5a614e9e13cb403f8f0b4cb199708089`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:38.681619 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-9b78b5e2-2b27-49c9-983a-ae627451e2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:38.749148 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1257/2474] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:38 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:38.766641 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433f36b-2a9a-4cf9-a2ef-d3003f8829f4 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:38.772588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1433f36b-2a9a-4cf9-a2ef-d3003f8829f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:38.831993 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433f36b-2a9a-4cf9-a2ef-d3003f8829f4 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:38.836488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3907a-2152-4e9e-a2c4-4f90f77e6afe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:38.837609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3907a-2152-4e9e-a2c4-4f90f77e6afe service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:38.837609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3907a-2152-4e9e-a2c4-4f90f77e6afe service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:38.837909 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3907a-2152-4e9e-a2c4-4f90f77e6afe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:38.848033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3907a-2152-4e9e-a2c4-4f90f77e6afe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:38.849464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3907a-2152-4e9e-a2c4-4f90f77e6afe service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:38.859057 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1218/2475] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:15:38 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:15:38.933207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78b5e2-2b27-49c9-983a-ae627451e2ee None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:38.943274 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78b5e2-2b27-49c9-983a-ae627451e2ee None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:38.946814 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d0b03-a929-43af-8408-006059d66f3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:38.947459 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d0b03-a929-43af-8408-006059d66f3c service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:38.947734 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d0b03-a929-43af-8408-006059d66f3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:38.948022 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d0b03-a929-43af-8408-006059d66f3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:39.004172 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d0b03-a929-43af-8408-006059d66f3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:39.005597 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d0b03-a929-43af-8408-006059d66f3c service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:39.014356 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1258/2476] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:15:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:15:46.400947 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3907a-2152-4e9e-a2c4-4f90f77e6afe service cinder] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:46.402455 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-41d3907a-2152-4e9e-a2c4-4f90f77e6afe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:46.412530 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:46.480627 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3907a-2152-4e9e-a2c4-4f90f77e6afe service cinder] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:46.485694 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8992817-7160-4463-81b7-2e334f34db60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:46.486835 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8992817-7160-4463-81b7-2e334f34db60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:46.487187 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8992817-7160-4463-81b7-2e334f34db60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:46.487454 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8992817-7160-4463-81b7-2e334f34db60 admin admin] PATH_INFO: `/v3/users/15155e8e1767492c88d540cb8d49f19f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:46.488591 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8992817-7160-4463-81b7-2e334f34db60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15155e8e1767492c88d540cb8d49f19f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:46.490611 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b1e78-3d79-4735-a5ba-6414b936de21 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:46.491191 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b1e78-3d79-4735-a5ba-6414b936de21 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:46.491191 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b1e78-3d79-4735-a5ba-6414b936de21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:46.496720 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8992817-7160-4463-81b7-2e334f34db60 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:46.521340 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-db5b1e78-3d79-4735-a5ba-6414b936de21 None None] MFA Rules not processed for user `76ebecce283f491e82a961d7f739878f`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:46.530970 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-db5b1e78-3d79-4735-a5ba-6414b936de21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:46.555080 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a8992817-7160-4463-81b7-2e334f34db60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15155e8e1767492c88d540cb8d49f19f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:46.555862 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a8992817-7160-4463-81b7-2e334f34db60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15155e8e1767492c88d540cb8d49f19f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:46.569525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a8992817-7160-4463-81b7-2e334f34db60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15155e8e1767492c88d540cb8d49f19f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:46.576504 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a8992817-7160-4463-81b7-2e334f34db60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15155e8e1767492c88d540cb8d49f19f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:46.577194 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a8992817-7160-4463-81b7-2e334f34db60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15155e8e1767492c88d540cb8d49f19f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:46.593037 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1219/2477] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:15:46 2022] DELETE /identity/v3/users/15155e8e1767492c88d540cb8d49f19f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:46.596410 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1259/2478] 10.222.0.36 () {58 vars in 967 bytes} [Fri Sep 2 09:15:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:15:46.597570 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf05367-81b7-4849-9dd4-c36f6ba64329 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:46.598223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf05367-81b7-4849-9dd4-c36f6ba64329 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:46.598400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf05367-81b7-4849-9dd4-c36f6ba64329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:46.612227 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b1e78-3d79-4735-a5ba-6414b936de21 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:46.614748 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-db5b1e78-3d79-4735-a5ba-6414b936de21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:46.619328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-1bf05367-81b7-4849-9dd4-c36f6ba64329 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:46.620583 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1bf05367-81b7-4849-9dd4-c36f6ba64329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:46.661935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:46.687120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1220/2479] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:15:46.691400 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b1e78-3d79-4735-a5ba-6414b936de21 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:46.697449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf05367-81b7-4849-9dd4-c36f6ba64329 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:46.698843 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1bf05367-81b7-4849-9dd4-c36f6ba64329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:46.731158 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:46.767954 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c51884-e749-4a5b-b54c-b060c9510fb9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92318948dcf1407b9fc8aa5c199787ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:46.768653 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c51884-e749-4a5b-b54c-b060c9510fb9 service swift] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:46.769011 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c51884-e749-4a5b-b54c-b060c9510fb9 service swift] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:46.769360 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c51884-e749-4a5b-b54c-b060c9510fb9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:46.777352 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf05367-81b7-4849-9dd4-c36f6ba64329 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:46.782243 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd17dba-4711-41e7-9f4e-8e82501166ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:46.782729 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd17dba-4711-41e7-9f4e-8e82501166ae service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:46.783042 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd17dba-4711-41e7-9f4e-8e82501166ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:46.783357 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd17dba-4711-41e7-9f4e-8e82501166ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:46.795557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd17dba-4711-41e7-9f4e-8e82501166ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:46.797016 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd17dba-4711-41e7-9f4e-8e82501166ae service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:46.814125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1221/2480] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:15:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:15:46.849029 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c51884-e749-4a5b-b54c-b060c9510fb9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:46.850976 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c51884-e749-4a5b-b54c-b060c9510fb9 service swift] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:46.858957 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1260/2481] 10.222.0.36 () {60 vars in 1384 bytes} [Fri Sep 2 09:15:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 02 09:15:47.115012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd17dba-4711-41e7-9f4e-8e82501166ae service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:47.116845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ebd17dba-4711-41e7-9f4e-8e82501166ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:47.136054 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd17dba-4711-41e7-9f4e-8e82501166ae service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:47.143694 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7a724-f82e-4dbc-9a66-f923126c3e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:47.144589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7a724-f82e-4dbc-9a66-f923126c3e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:47.144748 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7a724-f82e-4dbc-9a66-f923126c3e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:47.145075 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7a724-f82e-4dbc-9a66-f923126c3e06 admin admin] PATH_INFO: `/v3/projects/46068c98896a4b7e8e84a49488b6b6b6` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:47.155827 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7a724-f82e-4dbc-9a66-f923126c3e06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46068c98896a4b7e8e84a49488b6b6b6)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:47.157976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7a724-f82e-4dbc-9a66-f923126c3e06 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:47.206096 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-26b7a724-f82e-4dbc-9a66-f923126c3e06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46068c98896a4b7e8e84a49488b6b6b6', 'request_id': 'req-26b7a724-f82e-4dbc-9a66-f923126c3e06'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:47.213063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-26b7a724-f82e-4dbc-9a66-f923126c3e06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46068c98896a4b7e8e84a49488b6b6b6', 'request_id': 'req-26b7a724-f82e-4dbc-9a66-f923126c3e06'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:47.218809 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-26b7a724-f82e-4dbc-9a66-f923126c3e06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46068c98896a4b7e8e84a49488b6b6b6', 'request_id': 'req-26b7a724-f82e-4dbc-9a66-f923126c3e06'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:47.232412 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-26b7a724-f82e-4dbc-9a66-f923126c3e06 admin admin] The token cache is being invalidate because project 46068c98896a4b7e8e84a49488b6b6b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:15:47.232705 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-26b7a724-f82e-4dbc-9a66-f923126c3e06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:47.233410 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-26b7a724-f82e-4dbc-9a66-f923126c3e06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:47.234912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1222/2482] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:15:47 2022] DELETE /identity/v3/projects/46068c98896a4b7e8e84a49488b6b6b6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:47.239880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c51884-e749-4a5b-b54c-b060c9510fb9 service swift] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:47.250479 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c51884-e749-4a5b-b54c-b060c9510fb9 service swift] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:47.311200 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7defb04c-38b1-4f52-b218-8ac97c3ff2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:47.311893 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defb04c-38b1-4f52-b218-8ac97c3ff2a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:47.312187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defb04c-38b1-4f52-b218-8ac97c3ff2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:47.312549 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defb04c-38b1-4f52-b218-8ac97c3ff2a6 admin admin] PATH_INFO: `/v3/users/63a2c63f1bd24caca2aa70aa28219474` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:47.314127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7defb04c-38b1-4f52-b218-8ac97c3ff2a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63a2c63f1bd24caca2aa70aa28219474)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:47.316524 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7defb04c-38b1-4f52-b218-8ac97c3ff2a6 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:47.344452 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:47.366691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-7defb04c-38b1-4f52-b218-8ac97c3ff2a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63a2c63f1bd24caca2aa70aa28219474'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:47.376115 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-7defb04c-38b1-4f52-b218-8ac97c3ff2a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63a2c63f1bd24caca2aa70aa28219474'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:47.384601 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-7defb04c-38b1-4f52-b218-8ac97c3ff2a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63a2c63f1bd24caca2aa70aa28219474'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:47.385212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-7defb04c-38b1-4f52-b218-8ac97c3ff2a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63a2c63f1bd24caca2aa70aa28219474'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:47.385829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-7defb04c-38b1-4f52-b218-8ac97c3ff2a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63a2c63f1bd24caca2aa70aa28219474'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:47.403923 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1261/2483] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:15:47 2022] DELETE /identity/v3/users/63a2c63f1bd24caca2aa70aa28219474 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:47.703230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7a724-f82e-4dbc-9a66-f923126c3e06 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:47.704714 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-26b7a724-f82e-4dbc-9a66-f923126c3e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:47.728205 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7a724-f82e-4dbc-9a66-f923126c3e06 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:47.805577 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727aa13-694a-4898-b24e-5a934c65a415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:47.806494 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727aa13-694a-4898-b24e-5a934c65a415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:47.807366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727aa13-694a-4898-b24e-5a934c65a415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:47.807722 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727aa13-694a-4898-b24e-5a934c65a415 admin admin] PATH_INFO: `/v3/projects/250c3b68567644ddbecbb33740efe672` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:47.808954 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d727aa13-694a-4898-b24e-5a934c65a415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=250c3b68567644ddbecbb33740efe672)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:47.810845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d727aa13-694a-4898-b24e-5a934c65a415 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:47.875782 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d727aa13-694a-4898-b24e-5a934c65a415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '250c3b68567644ddbecbb33740efe672', 'request_id': 'req-d727aa13-694a-4898-b24e-5a934c65a415'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:47.883322 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d727aa13-694a-4898-b24e-5a934c65a415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '250c3b68567644ddbecbb33740efe672', 'request_id': 'req-d727aa13-694a-4898-b24e-5a934c65a415'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:47.884208 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:47.891946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d727aa13-694a-4898-b24e-5a934c65a415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '250c3b68567644ddbecbb33740efe672', 'request_id': 'req-d727aa13-694a-4898-b24e-5a934c65a415'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:47.912922 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d727aa13-694a-4898-b24e-5a934c65a415 admin admin] The token cache is being invalidate because project 250c3b68567644ddbecbb33740efe672 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:15:47.913443 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d727aa13-694a-4898-b24e-5a934c65a415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:47.914241 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d727aa13-694a-4898-b24e-5a934c65a415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:47.916074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1223/2484] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:15:47 2022] DELETE /identity/v3/projects/250c3b68567644ddbecbb33740efe672 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:47.921914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a12b8-9c50-4614-8314-793c492e3c6a None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:47.922406 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a12b8-9c50-4614-8314-793c492e3c6a None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:47.922552 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a12b8-9c50-4614-8314-793c492e3c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:47.943563 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-406a12b8-9c50-4614-8314-793c492e3c6a None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:47.944731 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-406a12b8-9c50-4614-8314-793c492e3c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:48.006420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1262/2485] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:48.014738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e6d18-6a18-469e-aae4-cff1f0a7305d None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:48.015053 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e6d18-6a18-469e-aae4-cff1f0a7305d None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:48.015353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e6d18-6a18-469e-aae4-cff1f0a7305d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:48.036294 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-6f4e6d18-6a18-469e-aae4-cff1f0a7305d None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:48.037877 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6f4e6d18-6a18-469e-aae4-cff1f0a7305d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:48.046159 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1224/2486] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:48.050391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a12b8-9c50-4614-8314-793c492e3c6a None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:48.061407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a12b8-9c50-4614-8314-793c492e3c6a None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:48.067269 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2cfc7-c3d8-47b2-b925-8110a61d7512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:48.067955 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2cfc7-c3d8-47b2-b925-8110a61d7512 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:48.068300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2cfc7-c3d8-47b2-b925-8110a61d7512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:48.068638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2cfc7-c3d8-47b2-b925-8110a61d7512 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:48.069350 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd2cfc7-c3d8-47b2-b925-8110a61d7512 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:48.069721 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd2cfc7-c3d8-47b2-b925-8110a61d7512 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:48.071438 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd2cfc7-c3d8-47b2-b925-8110a61d7512 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:48.087159 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1263/2487] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:48.088831 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e6d18-6a18-469e-aae4-cff1f0a7305d None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:48.098244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e6d18-6a18-469e-aae4-cff1f0a7305d None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:48.102821 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67319928-1ee9-419a-9255-729d82cec90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:48.103471 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67319928-1ee9-419a-9255-729d82cec90c admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:48.103750 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67319928-1ee9-419a-9255-729d82cec90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:48.104018 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67319928-1ee9-419a-9255-729d82cec90c admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:48.104651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67319928-1ee9-419a-9255-729d82cec90c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:48.106676 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67319928-1ee9-419a-9255-729d82cec90c admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:48.139155 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1225/2488] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:15:48 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:48.142026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2cfc7-c3d8-47b2-b925-8110a61d7512 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:48.152449 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2cfc7-c3d8-47b2-b925-8110a61d7512 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:48.216543 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4435b8-e7c7-49b8-ae24-a7b9ec7d5a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:48.217158 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4435b8-e7c7-49b8-ae24-a7b9ec7d5a4a admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:48.217442 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4435b8-e7c7-49b8-ae24-a7b9ec7d5a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:48.217710 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4435b8-e7c7-49b8-ae24-a7b9ec7d5a4a admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:48.218481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4435b8-e7c7-49b8-ae24-a7b9ec7d5a4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:48.220594 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4435b8-e7c7-49b8-ae24-a7b9ec7d5a4a admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:48.247883 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1264/2489] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:15:48 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:48.251839 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67319928-1ee9-419a-9255-729d82cec90c admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:48.265545 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67319928-1ee9-419a-9255-729d82cec90c admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:48.271104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f64cc-1269-4a1d-b242-0f6611a673d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:48.271744 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f64cc-1269-4a1d-b242-0f6611a673d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:48.272030 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f64cc-1269-4a1d-b242-0f6611a673d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:48.272303 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74f64cc-1269-4a1d-b242-0f6611a673d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:48.272976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f64cc-1269-4a1d-b242-0f6611a673d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:48.274542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74f64cc-1269-4a1d-b242-0f6611a673d4 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:48.287217 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1226/2490] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:48.290831 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4435b8-e7c7-49b8-ae24-a7b9ec7d5a4a admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:48.302628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4435b8-e7c7-49b8-ae24-a7b9ec7d5a4a admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:48.308193 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153081d5-56c1-4c09-b346-32f5fd9b1299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:48.308802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153081d5-56c1-4c09-b346-32f5fd9b1299 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:48.309103 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153081d5-56c1-4c09-b346-32f5fd9b1299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:48.309375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153081d5-56c1-4c09-b346-32f5fd9b1299 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:48.310020 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153081d5-56c1-4c09-b346-32f5fd9b1299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:48.311684 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153081d5-56c1-4c09-b346-32f5fd9b1299 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:48.329548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1265/2491] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:48.329768 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f64cc-1269-4a1d-b242-0f6611a673d4 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:48.343309 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74f64cc-1269-4a1d-b242-0f6611a673d4 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:48.354671 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77971918-6cce-418a-b41a-196a1bc26b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:48.355614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77971918-6cce-418a-b41a-196a1bc26b22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:48.356066 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77971918-6cce-418a-b41a-196a1bc26b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:48.356516 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77971918-6cce-418a-b41a-196a1bc26b22 admin admin] PATH_INFO: `/v3/projects/39741bba588d488f9ca657815d729437/users/23c7dc5188c44ca0a4717c313c1cde1f/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:48.377779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77971918-6cce-418a-b41a-196a1bc26b22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39741bba588d488f9ca657815d729437, user_id=23c7dc5188c44ca0a4717c313c1cde1f, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:48.380536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77971918-6cce-418a-b41a-196a1bc26b22 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:48.402259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1227/2492] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:15:48 2022] PUT /identity/v3/projects/39741bba588d488f9ca657815d729437/users/23c7dc5188c44ca0a4717c313c1cde1f/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:48.413298 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc27f95-0467-4bd2-b6f4-b1608aa5a789 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:48.413521 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc27f95-0467-4bd2-b6f4-b1608aa5a789 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:48.413893 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc27f95-0467-4bd2-b6f4-b1608aa5a789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:48.433712 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:48.444154 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-3bc27f95-0467-4bd2-b6f4-b1608aa5a789 None None] MFA Rules not processed for user `23c7dc5188c44ca0a4717c313c1cde1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:48.446203 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-3bc27f95-0467-4bd2-b6f4-b1608aa5a789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:48.510655 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1266/2493] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:48.522253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77971918-6cce-418a-b41a-196a1bc26b22 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:48.523975 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-77971918-6cce-418a-b41a-196a1bc26b22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:48.577988 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77971918-6cce-418a-b41a-196a1bc26b22 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:48.582677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5bb6d-13e5-4822-9fb4-817b1e7963c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:48.583299 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5bb6d-13e5-4822-9fb4-817b1e7963c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:48.583575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5bb6d-13e5-4822-9fb4-817b1e7963c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:48.583844 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5bb6d-13e5-4822-9fb4-817b1e7963c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:48.591929 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5bb6d-13e5-4822-9fb4-817b1e7963c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:48.593328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5bb6d-13e5-4822-9fb4-817b1e7963c3 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:48.602115 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1228/2494] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:15:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:15:49.086506 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc27f95-0467-4bd2-b6f4-b1608aa5a789 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:49.088101 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-3bc27f95-0467-4bd2-b6f4-b1608aa5a789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:49.154326 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc27f95-0467-4bd2-b6f4-b1608aa5a789 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:49.158940 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f60659a-0c18-414e-8daa-674df3c678ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:49.159662 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60659a-0c18-414e-8daa-674df3c678ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:49.159938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60659a-0c18-414e-8daa-674df3c678ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:49.160204 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f60659a-0c18-414e-8daa-674df3c678ea admin admin] PATH_INFO: `/v3/users/01a33cc6ec4c4c0589e7b556c90bc561` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:49.161335 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f60659a-0c18-414e-8daa-674df3c678ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=01a33cc6ec4c4c0589e7b556c90bc561)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:49.163269 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f60659a-0c18-414e-8daa-674df3c678ea admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:49.211019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9f60659a-0c18-414e-8daa-674df3c678ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01a33cc6ec4c4c0589e7b556c90bc561'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:49.217814 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9f60659a-0c18-414e-8daa-674df3c678ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01a33cc6ec4c4c0589e7b556c90bc561'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:49.226524 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9f60659a-0c18-414e-8daa-674df3c678ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01a33cc6ec4c4c0589e7b556c90bc561'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:49.227695 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9f60659a-0c18-414e-8daa-674df3c678ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01a33cc6ec4c4c0589e7b556c90bc561'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:49.228305 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9f60659a-0c18-414e-8daa-674df3c678ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01a33cc6ec4c4c0589e7b556c90bc561'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:49.261058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1267/2495] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:15:49 2022] DELETE /identity/v3/users/01a33cc6ec4c4c0589e7b556c90bc561 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:49.265664 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10be39-f85d-4fee-9e2f-375790c73bf1 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:49.266164 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10be39-f85d-4fee-9e2f-375790c73bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:49.266260 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c10be39-f85d-4fee-9e2f-375790c73bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:49.285457 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-6c10be39-f85d-4fee-9e2f-375790c73bf1 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:49.286663 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6c10be39-f85d-4fee-9e2f-375790c73bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:49.359114 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1229/2496] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:15:49.369977 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f60659a-0c18-414e-8daa-674df3c678ea admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:49.371997 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-9f60659a-0c18-414e-8daa-674df3c678ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:49.395301 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:49.439344 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f60659a-0c18-414e-8daa-674df3c678ea admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:49.443002 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fe6f0-df7d-42e6-8a0d-9c8da8c6304c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:49.443637 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fe6f0-df7d-42e6-8a0d-9c8da8c6304c service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:49.443917 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fe6f0-df7d-42e6-8a0d-9c8da8c6304c service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:49.444187 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fe6f0-df7d-42e6-8a0d-9c8da8c6304c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:49.454149 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fe6f0-df7d-42e6-8a0d-9c8da8c6304c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:49.455660 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fe6f0-df7d-42e6-8a0d-9c8da8c6304c service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:49.467655 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1268/2497] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:15:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:15:49.774053 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10be39-f85d-4fee-9e2f-375790c73bf1 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:49.776158 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6c10be39-f85d-4fee-9e2f-375790c73bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:49.790184 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c10be39-f85d-4fee-9e2f-375790c73bf1 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:49.807858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb53f02-b9d1-4346-9bf9-8ee4c06ee6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:49.808828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb53f02-b9d1-4346-9bf9-8ee4c06ee6b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:49.809210 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb53f02-b9d1-4346-9bf9-8ee4c06ee6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:49.809589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb53f02-b9d1-4346-9bf9-8ee4c06ee6b5 admin admin] PATH_INFO: `/v3/projects/e6d44d0c4b87445f95258b954e5f576d` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:49.810889 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb53f02-b9d1-4346-9bf9-8ee4c06ee6b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6d44d0c4b87445f95258b954e5f576d)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:49.812959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb53f02-b9d1-4346-9bf9-8ee4c06ee6b5 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:49.835923 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:49.870505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-feb53f02-b9d1-4346-9bf9-8ee4c06ee6b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6d44d0c4b87445f95258b954e5f576d', 'request_id': 'req-feb53f02-b9d1-4346-9bf9-8ee4c06ee6b5'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:49.877868 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-feb53f02-b9d1-4346-9bf9-8ee4c06ee6b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6d44d0c4b87445f95258b954e5f576d', 'request_id': 'req-feb53f02-b9d1-4346-9bf9-8ee4c06ee6b5'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:49.883878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-feb53f02-b9d1-4346-9bf9-8ee4c06ee6b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6d44d0c4b87445f95258b954e5f576d', 'request_id': 'req-feb53f02-b9d1-4346-9bf9-8ee4c06ee6b5'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:49.897753 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-feb53f02-b9d1-4346-9bf9-8ee4c06ee6b5 admin admin] The token cache is being invalidate because project e6d44d0c4b87445f95258b954e5f576d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:15:49.898046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-feb53f02-b9d1-4346-9bf9-8ee4c06ee6b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:49.898699 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-feb53f02-b9d1-4346-9bf9-8ee4c06ee6b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:49.900407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1230/2498] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:15:49 2022] DELETE /identity/v3/projects/e6d44d0c4b87445f95258b954e5f576d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:49.904434 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fe6f0-df7d-42e6-8a0d-9c8da8c6304c service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:49.914621 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fe6f0-df7d-42e6-8a0d-9c8da8c6304c service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:49.978130 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31002cd-9cca-4407-9675-8b5e9e9a8236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:49.978695 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31002cd-9cca-4407-9675-8b5e9e9a8236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:49.978971 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31002cd-9cca-4407-9675-8b5e9e9a8236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:49.979275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31002cd-9cca-4407-9675-8b5e9e9a8236 admin admin] PATH_INFO: `/v3/users/5a614e9e13cb403f8f0b4cb199708089` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:49.980374 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31002cd-9cca-4407-9675-8b5e9e9a8236 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a614e9e13cb403f8f0b4cb199708089)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:49.982553 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31002cd-9cca-4407-9675-8b5e9e9a8236 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:50.035756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e31002cd-9cca-4407-9675-8b5e9e9a8236 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a614e9e13cb403f8f0b4cb199708089'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:50.043011 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e31002cd-9cca-4407-9675-8b5e9e9a8236 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a614e9e13cb403f8f0b4cb199708089'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:50.052226 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e31002cd-9cca-4407-9675-8b5e9e9a8236 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a614e9e13cb403f8f0b4cb199708089'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:50.053048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e31002cd-9cca-4407-9675-8b5e9e9a8236 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a614e9e13cb403f8f0b4cb199708089'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:50.053728 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e31002cd-9cca-4407-9675-8b5e9e9a8236 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a614e9e13cb403f8f0b4cb199708089'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:50.071914 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1269/2499] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:15:49 2022] DELETE /identity/v3/users/5a614e9e13cb403f8f0b4cb199708089 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:50.388637 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb53f02-b9d1-4346-9bf9-8ee4c06ee6b5 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:50.390739 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-feb53f02-b9d1-4346-9bf9-8ee4c06ee6b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:50.405059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb53f02-b9d1-4346-9bf9-8ee4c06ee6b5 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:50.490070 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbde51d-73f4-46bf-9b0f-73afb92fb89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:50.490794 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbde51d-73f4-46bf-9b0f-73afb92fb89f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:50.491093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbde51d-73f4-46bf-9b0f-73afb92fb89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:50.491390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbde51d-73f4-46bf-9b0f-73afb92fb89f admin admin] PATH_INFO: `/v3/projects/8c43fb7a17314603b6addef503d297a6` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:50.492520 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbde51d-73f4-46bf-9b0f-73afb92fb89f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c43fb7a17314603b6addef503d297a6)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:50.494430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbde51d-73f4-46bf-9b0f-73afb92fb89f admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:50.539771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-ccbde51d-73f4-46bf-9b0f-73afb92fb89f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c43fb7a17314603b6addef503d297a6', 'request_id': 'req-ccbde51d-73f4-46bf-9b0f-73afb92fb89f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:50.545610 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-ccbde51d-73f4-46bf-9b0f-73afb92fb89f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c43fb7a17314603b6addef503d297a6', 'request_id': 'req-ccbde51d-73f4-46bf-9b0f-73afb92fb89f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:50.551966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-ccbde51d-73f4-46bf-9b0f-73afb92fb89f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c43fb7a17314603b6addef503d297a6', 'request_id': 'req-ccbde51d-73f4-46bf-9b0f-73afb92fb89f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:50.563659 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-ccbde51d-73f4-46bf-9b0f-73afb92fb89f admin admin] The token cache is being invalidate because project 8c43fb7a17314603b6addef503d297a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:15:50.563941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-ccbde51d-73f4-46bf-9b0f-73afb92fb89f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:50.565193 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-ccbde51d-73f4-46bf-9b0f-73afb92fb89f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:50.566689 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1231/2500] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:15:50 2022] DELETE /identity/v3/projects/8c43fb7a17314603b6addef503d297a6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:50.572504 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbc67b-62d6-492b-8553-913c4ad4e616 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:50.572990 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbc67b-62d6-492b-8553-913c4ad4e616 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:50.573098 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbc67b-62d6-492b-8553-913c4ad4e616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:50.592892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-62dbc67b-62d6-492b-8553-913c4ad4e616 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:50.594084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-62dbc67b-62d6-492b-8553-913c4ad4e616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:50.628967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:50.656287 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1270/2501] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:50.665299 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f486fa-deff-49a6-8fed-c0b09c854437 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:50.665580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f486fa-deff-49a6-8fed-c0b09c854437 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:50.665851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f486fa-deff-49a6-8fed-c0b09c854437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:50.685721 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-a2f486fa-deff-49a6-8fed-c0b09c854437 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:50.686921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-a2f486fa-deff-49a6-8fed-c0b09c854437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:50.695320 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1232/2502] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:50.699493 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbc67b-62d6-492b-8553-913c4ad4e616 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:50.709351 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbc67b-62d6-492b-8553-913c4ad4e616 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:50.714029 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef69a8c-3250-4e48-9731-81ef0c934561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:50.714621 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef69a8c-3250-4e48-9731-81ef0c934561 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:50.714898 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef69a8c-3250-4e48-9731-81ef0c934561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:50.715203 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef69a8c-3250-4e48-9731-81ef0c934561 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:50.715853 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef69a8c-3250-4e48-9731-81ef0c934561 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:50.716152 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef69a8c-3250-4e48-9731-81ef0c934561 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:50.717729 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef69a8c-3250-4e48-9731-81ef0c934561 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:50.729501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1271/2503] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:15:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:50.733602 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f486fa-deff-49a6-8fed-c0b09c854437 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:50.743321 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f486fa-deff-49a6-8fed-c0b09c854437 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:50.747806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94245c2-5f80-45b4-a6ef-2edcb14ac799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:50.748398 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94245c2-5f80-45b4-a6ef-2edcb14ac799 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:50.748671 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94245c2-5f80-45b4-a6ef-2edcb14ac799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:50.748937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94245c2-5f80-45b4-a6ef-2edcb14ac799 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:50.749559 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94245c2-5f80-45b4-a6ef-2edcb14ac799 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:50.751503 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94245c2-5f80-45b4-a6ef-2edcb14ac799 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:50.777510 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1233/2504] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:15:50 2022] POST /identity/v3/projects => generated 399 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:50.781491 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef69a8c-3250-4e48-9731-81ef0c934561 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:50.792142 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef69a8c-3250-4e48-9731-81ef0c934561 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:50.848773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45c154-cf68-4a73-ade2-2a8b51d5304f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:50.849376 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45c154-cf68-4a73-ade2-2a8b51d5304f admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:50.849651 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45c154-cf68-4a73-ade2-2a8b51d5304f admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:50.849960 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45c154-cf68-4a73-ade2-2a8b51d5304f admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:50.850567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45c154-cf68-4a73-ade2-2a8b51d5304f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:50.852479 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45c154-cf68-4a73-ade2-2a8b51d5304f admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:50.874801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1272/2505] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:15:50 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:50.878919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94245c2-5f80-45b4-a6ef-2edcb14ac799 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:50.889816 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94245c2-5f80-45b4-a6ef-2edcb14ac799 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:50.894735 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca5f6c-b7ff-42b0-ba7e-5a57b842241b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:50.895364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca5f6c-b7ff-42b0-ba7e-5a57b842241b admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:50.895657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca5f6c-b7ff-42b0-ba7e-5a57b842241b admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:50.895920 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ca5f6c-b7ff-42b0-ba7e-5a57b842241b admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:50.896563 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca5f6c-b7ff-42b0-ba7e-5a57b842241b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:50.898148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ca5f6c-b7ff-42b0-ba7e-5a57b842241b admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:50.899522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:50.911726 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1234/2506] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:50.915473 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45c154-cf68-4a73-ade2-2a8b51d5304f admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:50.928181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45c154-cf68-4a73-ade2-2a8b51d5304f admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:50.933540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2ab96f-09df-4e29-9d74-579020bcdd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:50.934358 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ab96f-09df-4e29-9d74-579020bcdd01 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:50.934641 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ab96f-09df-4e29-9d74-579020bcdd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:50.934919 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ab96f-09df-4e29-9d74-579020bcdd01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:50.935682 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ab96f-09df-4e29-9d74-579020bcdd01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:50.937296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2ab96f-09df-4e29-9d74-579020bcdd01 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:50.948333 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1273/2507] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:50.952526 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca5f6c-b7ff-42b0-ba7e-5a57b842241b admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:50.963038 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ca5f6c-b7ff-42b0-ba7e-5a57b842241b admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:50.969012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1f45c-fc51-432d-97f5-333b3da57e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:50.969691 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1f45c-fc51-432d-97f5-333b3da57e1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:50.969998 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1f45c-fc51-432d-97f5-333b3da57e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:50.970276 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1f45c-fc51-432d-97f5-333b3da57e1a admin admin] PATH_INFO: `/v3/projects/dda63d9dffe944f3a8975388493c7a8d/users/557b3747b21f4d36915abb8fef94ae6d/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:50.987664 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1f45c-fc51-432d-97f5-333b3da57e1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dda63d9dffe944f3a8975388493c7a8d, user_id=557b3747b21f4d36915abb8fef94ae6d, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:51.002752 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1f45c-fc51-432d-97f5-333b3da57e1a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:51.003580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1235/2508] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:15:50 2022] PUT /identity/v3/projects/dda63d9dffe944f3a8975388493c7a8d/users/557b3747b21f4d36915abb8fef94ae6d/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:15:51.013695 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd996549-3cac-4655-95b0-d775e8067d1b None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:51.014043 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd996549-3cac-4655-95b0-d775e8067d1b None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:51.014417 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd996549-3cac-4655-95b0-d775e8067d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:51.041358 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-fd996549-3cac-4655-95b0-d775e8067d1b None None] MFA Rules not processed for user `557b3747b21f4d36915abb8fef94ae6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:51.042561 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-fd996549-3cac-4655-95b0-d775e8067d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:51.105216 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1274/2509] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:51 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:51.109621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1f45c-fc51-432d-97f5-333b3da57e1a admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:51.170814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1f45c-fc51-432d-97f5-333b3da57e1a admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:51.175537 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48a092-d6aa-4921-abaf-a3ee0ec289cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:51.176137 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48a092-d6aa-4921-abaf-a3ee0ec289cd admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:51.176419 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48a092-d6aa-4921-abaf-a3ee0ec289cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:51.176689 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48a092-d6aa-4921-abaf-a3ee0ec289cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:51.177327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48a092-d6aa-4921-abaf-a3ee0ec289cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:51.179239 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c48a092-d6aa-4921-abaf-a3ee0ec289cd admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:51.204432 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1236/2510] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:15:51 2022] POST /identity/v3/projects => generated 401 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:15:51.209366 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd996549-3cac-4655-95b0-d775e8067d1b None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:51.219779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd996549-3cac-4655-95b0-d775e8067d1b None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:51.251595 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:51.276698 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4eec2f-5a3a-4b90-b6b5-9c1f547c5555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:51.277229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4eec2f-5a3a-4b90-b6b5-9c1f547c5555 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:51.277504 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4eec2f-5a3a-4b90-b6b5-9c1f547c5555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:51.277774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4eec2f-5a3a-4b90-b6b5-9c1f547c5555 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:51.278393 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4eec2f-5a3a-4b90-b6b5-9c1f547c5555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:51.280338 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4eec2f-5a3a-4b90-b6b5-9c1f547c5555 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:51.300902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1275/2511] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:15:51 2022] POST /identity/v3/users => generated 307 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:51.304944 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48a092-d6aa-4921-abaf-a3ee0ec289cd admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:51.315395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48a092-d6aa-4921-abaf-a3ee0ec289cd admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:51.320146 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f92ef-5801-48fb-bb74-6c1e52916133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:51.320755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f92ef-5801-48fb-bb74-6c1e52916133 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:51.321056 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f92ef-5801-48fb-bb74-6c1e52916133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:51.321334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f92ef-5801-48fb-bb74-6c1e52916133 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:51.321983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f92ef-5801-48fb-bb74-6c1e52916133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:51.323632 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f92ef-5801-48fb-bb74-6c1e52916133 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:51.335328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1237/2512] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:51.339655 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4eec2f-5a3a-4b90-b6b5-9c1f547c5555 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:51.350404 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4eec2f-5a3a-4b90-b6b5-9c1f547c5555 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:51.355026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996bf236-8735-4163-9fa1-0230a2e7a360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:51.355764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996bf236-8735-4163-9fa1-0230a2e7a360 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:51.356064 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996bf236-8735-4163-9fa1-0230a2e7a360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:51.356946 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996bf236-8735-4163-9fa1-0230a2e7a360 admin admin] PATH_INFO: `/v3/projects/e86ee95eebf24d8aae0ab3f55419c735/users/7941a701d0c448f3b5220fbe474f9285/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:51.372420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996bf236-8735-4163-9fa1-0230a2e7a360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e86ee95eebf24d8aae0ab3f55419c735, user_id=7941a701d0c448f3b5220fbe474f9285, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:51.375177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996bf236-8735-4163-9fa1-0230a2e7a360 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:51.388715 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1276/2513] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:15:51 2022] PUT /identity/v3/projects/e86ee95eebf24d8aae0ab3f55419c735/users/7941a701d0c448f3b5220fbe474f9285/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:51.393430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f92ef-5801-48fb-bb74-6c1e52916133 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:51.405126 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f92ef-5801-48fb-bb74-6c1e52916133 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:51.463806 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157f898-57b6-459a-ae2e-19024bd0af17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:51.464413 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157f898-57b6-459a-ae2e-19024bd0af17 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:51.464690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157f898-57b6-459a-ae2e-19024bd0af17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:51.464958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157f898-57b6-459a-ae2e-19024bd0af17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:51.465724 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1157f898-57b6-459a-ae2e-19024bd0af17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:51.467359 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1157f898-57b6-459a-ae2e-19024bd0af17 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:51.478219 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1238/2514] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:15:51.482714 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996bf236-8735-4163-9fa1-0230a2e7a360 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:51.495573 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996bf236-8735-4163-9fa1-0230a2e7a360 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:51.500671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217090f-89cc-41b1-b232-f06ab842cd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:51.501405 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217090f-89cc-41b1-b232-f06ab842cd36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:51.502244 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217090f-89cc-41b1-b232-f06ab842cd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:51.502592 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217090f-89cc-41b1-b232-f06ab842cd36 admin admin] PATH_INFO: `/v3/projects/e86ee95eebf24d8aae0ab3f55419c735/users/7941a701d0c448f3b5220fbe474f9285/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:51.504756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217090f-89cc-41b1-b232-f06ab842cd36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e86ee95eebf24d8aae0ab3f55419c735, user_id=7941a701d0c448f3b5220fbe474f9285, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:51.507842 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217090f-89cc-41b1-b232-f06ab842cd36 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:51.520780 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1277/2515] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:15:51 2022] PUT /identity/v3/projects/e86ee95eebf24d8aae0ab3f55419c735/users/7941a701d0c448f3b5220fbe474f9285/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:51.537737 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d5a21-8ed0-4692-a090-38e5da1d83a7 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:51.537737 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d5a21-8ed0-4692-a090-38e5da1d83a7 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:51.537737 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d5a21-8ed0-4692-a090-38e5da1d83a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:51.568910 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-448d5a21-8ed0-4692-a090-38e5da1d83a7 None None] MFA Rules not processed for user `7941a701d0c448f3b5220fbe474f9285`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:51.570412 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-448d5a21-8ed0-4692-a090-38e5da1d83a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:51.606349 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:51.652677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1239/2516] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:51 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:51.665567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217090f-89cc-41b1-b232-f06ab842cd36 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:51.668750 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-9217090f-89cc-41b1-b232-f06ab842cd36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:51.746153 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217090f-89cc-41b1-b232-f06ab842cd36 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:51.749799 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37286448-cc05-4746-8e9d-a2de898a330b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:51.750450 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37286448-cc05-4746-8e9d-a2de898a330b service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:51.750730 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37286448-cc05-4746-8e9d-a2de898a330b service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:51.750995 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37286448-cc05-4746-8e9d-a2de898a330b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:51.760960 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37286448-cc05-4746-8e9d-a2de898a330b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:51.762391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37286448-cc05-4746-8e9d-a2de898a330b service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:51.773185 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1278/2517] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:15:51 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:15:51.850238 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d5a21-8ed0-4692-a090-38e5da1d83a7 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:51.859952 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d5a21-8ed0-4692-a090-38e5da1d83a7 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:51.864202 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f4d61-9764-4089-a322-0eaebbcb3baf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:51.864886 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f4d61-9764-4089-a322-0eaebbcb3baf service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:51.865145 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f4d61-9764-4089-a322-0eaebbcb3baf service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:51.865413 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f4d61-9764-4089-a322-0eaebbcb3baf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:51.930994 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f4d61-9764-4089-a322-0eaebbcb3baf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:51.932477 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f4d61-9764-4089-a322-0eaebbcb3baf service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:51.942461 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1240/2518] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:15:51 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:15:56.085479 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37286448-cc05-4746-8e9d-a2de898a330b service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:56.145100 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37286448-cc05-4746-8e9d-a2de898a330b service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:56.149558 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269881cb-7c51-47dd-91ac-f8afe42b4a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:56.150172 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269881cb-7c51-47dd-91ac-f8afe42b4a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:56.150449 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269881cb-7c51-47dd-91ac-f8afe42b4a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:56.150719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269881cb-7c51-47dd-91ac-f8afe42b4a53 admin admin] PATH_INFO: `/v3/users/557b3747b21f4d36915abb8fef94ae6d` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:56.151854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269881cb-7c51-47dd-91ac-f8afe42b4a53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=557b3747b21f4d36915abb8fef94ae6d)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:56.153755 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269881cb-7c51-47dd-91ac-f8afe42b4a53 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:56.186237 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:56.199560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-269881cb-7c51-47dd-91ac-f8afe42b4a53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '557b3747b21f4d36915abb8fef94ae6d'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:56.207140 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-269881cb-7c51-47dd-91ac-f8afe42b4a53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '557b3747b21f4d36915abb8fef94ae6d'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:56.214412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-269881cb-7c51-47dd-91ac-f8afe42b4a53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '557b3747b21f4d36915abb8fef94ae6d'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:56.215035 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-269881cb-7c51-47dd-91ac-f8afe42b4a53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '557b3747b21f4d36915abb8fef94ae6d'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:56.215702 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-269881cb-7c51-47dd-91ac-f8afe42b4a53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '557b3747b21f4d36915abb8fef94ae6d'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:56.232548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1279/2519] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:15:56 2022] DELETE /identity/v3/users/557b3747b21f4d36915abb8fef94ae6d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:56.237007 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47047d29-909a-4a0d-a1ff-698813a009f9 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:56.237545 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47047d29-909a-4a0d-a1ff-698813a009f9 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:56.237645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47047d29-909a-4a0d-a1ff-698813a009f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:56.256840 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-47047d29-909a-4a0d-a1ff-698813a009f9 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:56.258036 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-47047d29-909a-4a0d-a1ff-698813a009f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:56.316021 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1241/2520] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:56.325890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269881cb-7c51-47dd-91ac-f8afe42b4a53 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:56.327328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-269881cb-7c51-47dd-91ac-f8afe42b4a53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:56.387864 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269881cb-7c51-47dd-91ac-f8afe42b4a53 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:56.391712 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1133b9-881a-4df1-8559-d742f90f325b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:56.392470 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1133b9-881a-4df1-8559-d742f90f325b service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:56.392872 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1133b9-881a-4df1-8559-d742f90f325b service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:56.393265 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1133b9-881a-4df1-8559-d742f90f325b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:56.403347 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1133b9-881a-4df1-8559-d742f90f325b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:56.404778 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1133b9-881a-4df1-8559-d742f90f325b service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:56.418229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1280/2521] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:15:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:15:56.703802 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47047d29-909a-4a0d-a1ff-698813a009f9 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:56.705229 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-47047d29-909a-4a0d-a1ff-698813a009f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:56.717908 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47047d29-909a-4a0d-a1ff-698813a009f9 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:56.722632 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805510f-4e9d-4ce1-9275-81a9be7cd1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:56.723302 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805510f-4e9d-4ce1-9275-81a9be7cd1d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:56.723570 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805510f-4e9d-4ce1-9275-81a9be7cd1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:56.723838 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805510f-4e9d-4ce1-9275-81a9be7cd1d8 admin admin] PATH_INFO: `/v3/projects/dda63d9dffe944f3a8975388493c7a8d` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:56.724944 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805510f-4e9d-4ce1-9275-81a9be7cd1d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dda63d9dffe944f3a8975388493c7a8d)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:56.726835 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a805510f-4e9d-4ce1-9275-81a9be7cd1d8 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:56.773941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a805510f-4e9d-4ce1-9275-81a9be7cd1d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dda63d9dffe944f3a8975388493c7a8d', 'request_id': 'req-a805510f-4e9d-4ce1-9275-81a9be7cd1d8'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:56.779881 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a805510f-4e9d-4ce1-9275-81a9be7cd1d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dda63d9dffe944f3a8975388493c7a8d', 'request_id': 'req-a805510f-4e9d-4ce1-9275-81a9be7cd1d8'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:56.784388 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:56.785599 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a805510f-4e9d-4ce1-9275-81a9be7cd1d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dda63d9dffe944f3a8975388493c7a8d', 'request_id': 'req-a805510f-4e9d-4ce1-9275-81a9be7cd1d8'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:56.799339 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a805510f-4e9d-4ce1-9275-81a9be7cd1d8 admin admin] The token cache is being invalidate because project dda63d9dffe944f3a8975388493c7a8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:15:56.799625 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a805510f-4e9d-4ce1-9275-81a9be7cd1d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:56.800732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a805510f-4e9d-4ce1-9275-81a9be7cd1d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:56.802187 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1242/2522] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:15:56 2022] DELETE /identity/v3/projects/dda63d9dffe944f3a8975388493c7a8d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:56.806088 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1133b9-881a-4df1-8559-d742f90f325b service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:56.815734 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1133b9-881a-4df1-8559-d742f90f325b service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:56.871266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18758a0-1a5f-45b9-8f8a-39abe55bf61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:56.871902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18758a0-1a5f-45b9-8f8a-39abe55bf61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:56.872182 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18758a0-1a5f-45b9-8f8a-39abe55bf61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:56.872454 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18758a0-1a5f-45b9-8f8a-39abe55bf61e admin admin] PATH_INFO: `/v3/users/7941a701d0c448f3b5220fbe474f9285` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:56.873557 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18758a0-1a5f-45b9-8f8a-39abe55bf61e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7941a701d0c448f3b5220fbe474f9285)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:56.875520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18758a0-1a5f-45b9-8f8a-39abe55bf61e admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:56.920343 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e18758a0-1a5f-45b9-8f8a-39abe55bf61e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7941a701d0c448f3b5220fbe474f9285'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:56.925704 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:56.927825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e18758a0-1a5f-45b9-8f8a-39abe55bf61e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7941a701d0c448f3b5220fbe474f9285'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:56.935540 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e18758a0-1a5f-45b9-8f8a-39abe55bf61e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7941a701d0c448f3b5220fbe474f9285'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:56.936248 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e18758a0-1a5f-45b9-8f8a-39abe55bf61e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7941a701d0c448f3b5220fbe474f9285'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:56.936988 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e18758a0-1a5f-45b9-8f8a-39abe55bf61e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7941a701d0c448f3b5220fbe474f9285'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:56.953994 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1281/2523] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:15:56 2022] DELETE /identity/v3/users/7941a701d0c448f3b5220fbe474f9285 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:57.221061 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805510f-4e9d-4ce1-9275-81a9be7cd1d8 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:57.223031 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-a805510f-4e9d-4ce1-9275-81a9be7cd1d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:57.236785 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a805510f-4e9d-4ce1-9275-81a9be7cd1d8 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:57.295481 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79131ec5-c76c-475f-a7d6-39a8f53f873d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:57.296197 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79131ec5-c76c-475f-a7d6-39a8f53f873d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:57.296388 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79131ec5-c76c-475f-a7d6-39a8f53f873d admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:57.296663 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79131ec5-c76c-475f-a7d6-39a8f53f873d admin admin] PATH_INFO: `/v3/projects/e86ee95eebf24d8aae0ab3f55419c735` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:57.297834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79131ec5-c76c-475f-a7d6-39a8f53f873d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e86ee95eebf24d8aae0ab3f55419c735)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:57.299839 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79131ec5-c76c-475f-a7d6-39a8f53f873d admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:57.348575 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-79131ec5-c76c-475f-a7d6-39a8f53f873d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e86ee95eebf24d8aae0ab3f55419c735', 'request_id': 'req-79131ec5-c76c-475f-a7d6-39a8f53f873d'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:57.354843 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-79131ec5-c76c-475f-a7d6-39a8f53f873d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e86ee95eebf24d8aae0ab3f55419c735', 'request_id': 'req-79131ec5-c76c-475f-a7d6-39a8f53f873d'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:57.360490 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-79131ec5-c76c-475f-a7d6-39a8f53f873d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e86ee95eebf24d8aae0ab3f55419c735', 'request_id': 'req-79131ec5-c76c-475f-a7d6-39a8f53f873d'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:57.373952 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-79131ec5-c76c-475f-a7d6-39a8f53f873d admin admin] The token cache is being invalidate because project e86ee95eebf24d8aae0ab3f55419c735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:15:57.374368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-79131ec5-c76c-475f-a7d6-39a8f53f873d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:57.375012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-79131ec5-c76c-475f-a7d6-39a8f53f873d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:15:57.376655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1243/2524] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:15:57 2022] DELETE /identity/v3/projects/e86ee95eebf24d8aae0ab3f55419c735 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:57.381558 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa23600-22d5-4a39-8340-ce583f7de8d9 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:57.382039 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa23600-22d5-4a39-8340-ce583f7de8d9 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:57.382139 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa23600-22d5-4a39-8340-ce583f7de8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:57.401272 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-2fa23600-22d5-4a39-8340-ce583f7de8d9 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:57.402471 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-2fa23600-22d5-4a39-8340-ce583f7de8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:57.461350 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1282/2525] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:57.470319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0e198-2e3f-45d2-8997-55ddfe80a1d8 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:57.470540 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0e198-2e3f-45d2-8997-55ddfe80a1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:57.470810 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0e198-2e3f-45d2-8997-55ddfe80a1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:57.491074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-04c0e198-2e3f-45d2-8997-55ddfe80a1d8 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:57.492363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-04c0e198-2e3f-45d2-8997-55ddfe80a1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:57.500244 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1244/2526] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:57.504489 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa23600-22d5-4a39-8340-ce583f7de8d9 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:57.514206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa23600-22d5-4a39-8340-ce583f7de8d9 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:57.519319 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccec4fa-da46-417c-b64b-a63910c82a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:57.519910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccec4fa-da46-417c-b64b-a63910c82a90 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:57.520191 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccec4fa-da46-417c-b64b-a63910c82a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:57.520464 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccec4fa-da46-417c-b64b-a63910c82a90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:57.521101 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccec4fa-da46-417c-b64b-a63910c82a90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:57.521405 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccec4fa-da46-417c-b64b-a63910c82a90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:57.523012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccec4fa-da46-417c-b64b-a63910c82a90 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:57.534579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1283/2527] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:57.538790 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0e198-2e3f-45d2-8997-55ddfe80a1d8 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:57.547851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0e198-2e3f-45d2-8997-55ddfe80a1d8 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:57.552972 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6bbb8-4ea4-4d65-96fd-a5c1f219245f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:57.553560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6bbb8-4ea4-4d65-96fd-a5c1f219245f admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:57.553840 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6bbb8-4ea4-4d65-96fd-a5c1f219245f admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:57.554112 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6bbb8-4ea4-4d65-96fd-a5c1f219245f admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:57.554746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde6bbb8-4ea4-4d65-96fd-a5c1f219245f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:57.556660 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde6bbb8-4ea4-4d65-96fd-a5c1f219245f admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:57.584190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1245/2528] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:15:57 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:57.588757 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccec4fa-da46-417c-b64b-a63910c82a90 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:57.598739 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccec4fa-da46-417c-b64b-a63910c82a90 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:57.662173 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42824c0d-b981-406c-b8b3-ab334a64ea91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:57.662782 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42824c0d-b981-406c-b8b3-ab334a64ea91 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:57.663061 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42824c0d-b981-406c-b8b3-ab334a64ea91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:57.663364 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42824c0d-b981-406c-b8b3-ab334a64ea91 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:57.664014 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42824c0d-b981-406c-b8b3-ab334a64ea91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:57.666076 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42824c0d-b981-406c-b8b3-ab334a64ea91 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:57.687481 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1284/2529] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:15:57 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:57.692185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6bbb8-4ea4-4d65-96fd-a5c1f219245f admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:57.702226 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6bbb8-4ea4-4d65-96fd-a5c1f219245f admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:57.707092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de228a4-8ffa-464f-b455-f58f95b1b424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:57.707700 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de228a4-8ffa-464f-b455-f58f95b1b424 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:57.707977 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de228a4-8ffa-464f-b455-f58f95b1b424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:57.708248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de228a4-8ffa-464f-b455-f58f95b1b424 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:57.708915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de228a4-8ffa-464f-b455-f58f95b1b424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:57.710488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de228a4-8ffa-464f-b455-f58f95b1b424 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:57.720275 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1246/2530] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:57.724044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42824c0d-b981-406c-b8b3-ab334a64ea91 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:57.733699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42824c0d-b981-406c-b8b3-ab334a64ea91 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:57.738813 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c52a3ba-9130-4a02-a5b2-34d188563a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:57.739430 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52a3ba-9130-4a02-a5b2-34d188563a61 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:57.739707 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52a3ba-9130-4a02-a5b2-34d188563a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:57.739975 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c52a3ba-9130-4a02-a5b2-34d188563a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:57.740596 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52a3ba-9130-4a02-a5b2-34d188563a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:57.742170 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c52a3ba-9130-4a02-a5b2-34d188563a61 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:57.752439 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1285/2531] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:57.757125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de228a4-8ffa-464f-b455-f58f95b1b424 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:57.766919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de228a4-8ffa-464f-b455-f58f95b1b424 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:57.771590 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda9298-f6b7-43a2-a7f6-e924c7494406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:57.772227 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda9298-f6b7-43a2-a7f6-e924c7494406 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:57.772506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda9298-f6b7-43a2-a7f6-e924c7494406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:57.772777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda9298-f6b7-43a2-a7f6-e924c7494406 admin admin] PATH_INFO: `/v3/projects/cfaae54fcadd43fa8a1f3f7e224519e8/users/c0568b9d4c8e4503a2c4613c76a3d63c/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:57.786983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda9298-f6b7-43a2-a7f6-e924c7494406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfaae54fcadd43fa8a1f3f7e224519e8, user_id=c0568b9d4c8e4503a2c4613c76a3d63c, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:57.789645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda9298-f6b7-43a2-a7f6-e924c7494406 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:57.802213 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1247/2532] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:15:57 2022] PUT /identity/v3/projects/cfaae54fcadd43fa8a1f3f7e224519e8/users/c0568b9d4c8e4503a2c4613c76a3d63c/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:57.812215 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15437b-8855-4371-86aa-6ad789a6e0e5 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:57.812501 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15437b-8855-4371-86aa-6ad789a6e0e5 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:57.812774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15437b-8855-4371-86aa-6ad789a6e0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:57.840852 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-6f15437b-8855-4371-86aa-6ad789a6e0e5 None None] MFA Rules not processed for user `c0568b9d4c8e4503a2c4613c76a3d63c`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:57.842076 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6f15437b-8855-4371-86aa-6ad789a6e0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:57.867830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:57.906605 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1286/2533] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:57.910998 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda9298-f6b7-43a2-a7f6-e924c7494406 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:57.928175 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:57.972865 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda9298-f6b7-43a2-a7f6-e924c7494406 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:57.977491 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00a9b2-3bd2-4efa-86ef-3a7a521449c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:57.978062 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00a9b2-3bd2-4efa-86ef-3a7a521449c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:57.978340 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00a9b2-3bd2-4efa-86ef-3a7a521449c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:57.978613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00a9b2-3bd2-4efa-86ef-3a7a521449c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:57.979271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00a9b2-3bd2-4efa-86ef-3a7a521449c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:57.981154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00a9b2-3bd2-4efa-86ef-3a7a521449c3 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:58.007086 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1248/2534] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:15:57 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:15:58.011139 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15437b-8855-4371-86aa-6ad789a6e0e5 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:58.022071 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15437b-8855-4371-86aa-6ad789a6e0e5 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:58.077855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d81192-b937-450a-88b4-a9d19a0c7440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:58.078456 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d81192-b937-450a-88b4-a9d19a0c7440 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:58.078736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d81192-b937-450a-88b4-a9d19a0c7440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:58.079011 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d81192-b937-450a-88b4-a9d19a0c7440 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:58.079670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d81192-b937-450a-88b4-a9d19a0c7440 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:58.081581 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d81192-b937-450a-88b4-a9d19a0c7440 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:58.101924 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1287/2535] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:15:58 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:15:58.105867 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00a9b2-3bd2-4efa-86ef-3a7a521449c3 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:58.116504 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00a9b2-3bd2-4efa-86ef-3a7a521449c3 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:58.121097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a97db-8f23-4915-8be2-243f4e11a31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:58.121690 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a97db-8f23-4915-8be2-243f4e11a31c admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:58.121966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a97db-8f23-4915-8be2-243f4e11a31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:58.122239 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a97db-8f23-4915-8be2-243f4e11a31c admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:58.122918 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a97db-8f23-4915-8be2-243f4e11a31c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:58.124558 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a97db-8f23-4915-8be2-243f4e11a31c admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:58.134839 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1249/2536] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:58.138697 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d81192-b937-450a-88b4-a9d19a0c7440 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:58.148574 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d81192-b937-450a-88b4-a9d19a0c7440 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:58.153148 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f43bd-3020-4cb1-9055-c706fada1fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:58.153783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f43bd-3020-4cb1-9055-c706fada1fae admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:58.154065 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f43bd-3020-4cb1-9055-c706fada1fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:58.154338 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1f43bd-3020-4cb1-9055-c706fada1fae admin admin] PATH_INFO: `/v3/projects/c6fc792d15354a7aa22f9a3fedd29951/users/41f92c5167794d849247c0d6e436cde5/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:58.169336 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f43bd-3020-4cb1-9055-c706fada1fae admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6fc792d15354a7aa22f9a3fedd29951, user_id=41f92c5167794d849247c0d6e436cde5, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:58.172038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1f43bd-3020-4cb1-9055-c706fada1fae admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:58.185152 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1288/2537] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:15:58 2022] PUT /identity/v3/projects/c6fc792d15354a7aa22f9a3fedd29951/users/41f92c5167794d849247c0d6e436cde5/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:58.188940 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a97db-8f23-4915-8be2-243f4e11a31c admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:58.198829 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a97db-8f23-4915-8be2-243f4e11a31c admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:58.253939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e273db3-30b4-4488-9e66-807db1ce8eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:58.254462 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e273db3-30b4-4488-9e66-807db1ce8eda admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:58.254765 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e273db3-30b4-4488-9e66-807db1ce8eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:58.255033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e273db3-30b4-4488-9e66-807db1ce8eda admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:58.255765 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e273db3-30b4-4488-9e66-807db1ce8eda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:58.257418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e273db3-30b4-4488-9e66-807db1ce8eda admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:58.266839 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1250/2538] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:58.271602 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f43bd-3020-4cb1-9055-c706fada1fae admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:58.282025 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1f43bd-3020-4cb1-9055-c706fada1fae admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:58.287323 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9041d-cad2-4692-94f8-89c40a9581f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:58.287955 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9041d-cad2-4692-94f8-89c40a9581f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:58.288236 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9041d-cad2-4692-94f8-89c40a9581f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:58.288507 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9041d-cad2-4692-94f8-89c40a9581f7 admin admin] PATH_INFO: `/v3/projects/c6fc792d15354a7aa22f9a3fedd29951/users/41f92c5167794d849247c0d6e436cde5/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:58.290420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9041d-cad2-4692-94f8-89c40a9581f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6fc792d15354a7aa22f9a3fedd29951, user_id=41f92c5167794d849247c0d6e436cde5, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:58.293080 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9041d-cad2-4692-94f8-89c40a9581f7 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:58.303151 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1289/2539] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:15:58 2022] PUT /identity/v3/projects/c6fc792d15354a7aa22f9a3fedd29951/users/41f92c5167794d849247c0d6e436cde5/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:58.314063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d447ad-fef8-432c-a31c-2a4b6505c939 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:58.314527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d447ad-fef8-432c-a31c-2a4b6505c939 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:58.314621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d447ad-fef8-432c-a31c-2a4b6505c939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:58.342039 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-66d447ad-fef8-432c-a31c-2a4b6505c939 None None] MFA Rules not processed for user `41f92c5167794d849247c0d6e436cde5`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:58.343302 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-66d447ad-fef8-432c-a31c-2a4b6505c939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:58.412395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1251/2540] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:15:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:15:58.423092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9041d-cad2-4692-94f8-89c40a9581f7 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:58.425373 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-22e9041d-cad2-4692-94f8-89c40a9581f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:58.482326 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9041d-cad2-4692-94f8-89c40a9581f7 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:58.486036 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbf867-6673-4f81-b973-6ae3328f9d1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:58.486633 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbf867-6673-4f81-b973-6ae3328f9d1c service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:58.486916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbf867-6673-4f81-b973-6ae3328f9d1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:58.487816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbf867-6673-4f81-b973-6ae3328f9d1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:58.497126 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcbf867-6673-4f81-b973-6ae3328f9d1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:58.498628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcbf867-6673-4f81-b973-6ae3328f9d1c service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:58.508121 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1290/2541] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:15:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:15:58.608026 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d447ad-fef8-432c-a31c-2a4b6505c939 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:58.618031 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d447ad-fef8-432c-a31c-2a4b6505c939 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:58.621733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735b1c5-5e9b-4303-8057-f94eb962480e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:58.622362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735b1c5-5e9b-4303-8057-f94eb962480e service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:58.622638 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735b1c5-5e9b-4303-8057-f94eb962480e service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:58.622905 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735b1c5-5e9b-4303-8057-f94eb962480e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:58.631628 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:58.681268 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735b1c5-5e9b-4303-8057-f94eb962480e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:58.682678 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735b1c5-5e9b-4303-8057-f94eb962480e service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:58.692351 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1252/2542] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:15:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:15:59.813177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbf867-6673-4f81-b973-6ae3328f9d1c service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:59.814642 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-9fcbf867-6673-4f81-b973-6ae3328f9d1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:59.843338 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:59.876749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbf867-6673-4f81-b973-6ae3328f9d1c service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:59.880576 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208a3a8-6b7b-43b9-bfa9-cb3c0bc22766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:59.881183 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208a3a8-6b7b-43b9-bfa9-cb3c0bc22766 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:59.881462 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208a3a8-6b7b-43b9-bfa9-cb3c0bc22766 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:59.881755 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208a3a8-6b7b-43b9-bfa9-cb3c0bc22766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:59.883487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-9208a3a8-6b7b-43b9-bfa9-cb3c0bc22766 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:59.949088 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208a3a8-6b7b-43b9-bfa9-cb3c0bc22766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:59.950555 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208a3a8-6b7b-43b9-bfa9-cb3c0bc22766 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:59.960922 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1291/2543] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:15:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:16:01.282555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735b1c5-5e9b-4303-8057-f94eb962480e service cinder] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:01.298736 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735b1c5-5e9b-4303-8057-f94eb962480e service cinder] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:01.304143 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad59f9-985e-4fa9-bb90-80e7691e9730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:01.304782 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad59f9-985e-4fa9-bb90-80e7691e9730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:01.305062 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad59f9-985e-4fa9-bb90-80e7691e9730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:01.305331 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad59f9-985e-4fa9-bb90-80e7691e9730 admin admin] PATH_INFO: `/v3/users/c0568b9d4c8e4503a2c4613c76a3d63c` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:01.306535 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad59f9-985e-4fa9-bb90-80e7691e9730 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0568b9d4c8e4503a2c4613c76a3d63c)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:01.308385 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad59f9-985e-4fa9-bb90-80e7691e9730 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:01.398018 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-15ad59f9-985e-4fa9-bb90-80e7691e9730 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0568b9d4c8e4503a2c4613c76a3d63c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:01.398901 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-15ad59f9-985e-4fa9-bb90-80e7691e9730 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0568b9d4c8e4503a2c4613c76a3d63c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:01.428926 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-15ad59f9-985e-4fa9-bb90-80e7691e9730 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0568b9d4c8e4503a2c4613c76a3d63c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:01.452894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-15ad59f9-985e-4fa9-bb90-80e7691e9730 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0568b9d4c8e4503a2c4613c76a3d63c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:01.454675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-15ad59f9-985e-4fa9-bb90-80e7691e9730 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0568b9d4c8e4503a2c4613c76a3d63c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:01.478594 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1253/2544] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:01 2022] DELETE /identity/v3/users/c0568b9d4c8e4503a2c4613c76a3d63c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:01.480224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1edede-0736-4f05-b4f8-ca0e88504850 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:01.480724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1edede-0736-4f05-b4f8-ca0e88504850 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:01.480869 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1edede-0736-4f05-b4f8-ca0e88504850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:01.505794 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-0e1edede-0736-4f05-b4f8-ca0e88504850 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:01.507626 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0e1edede-0736-4f05-b4f8-ca0e88504850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:01.584105 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1292/2545] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:01.593646 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad59f9-985e-4fa9-bb90-80e7691e9730 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:01.596212 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-15ad59f9-985e-4fa9-bb90-80e7691e9730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:01.661322 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad59f9-985e-4fa9-bb90-80e7691e9730 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:01.665146 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374689a2-d324-46fe-a25f-08516ba843a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:01.665881 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374689a2-d324-46fe-a25f-08516ba843a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:01.666365 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374689a2-d324-46fe-a25f-08516ba843a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:01.666755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374689a2-d324-46fe-a25f-08516ba843a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:01.676401 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374689a2-d324-46fe-a25f-08516ba843a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:01.678431 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374689a2-d324-46fe-a25f-08516ba843a8 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:01.695275 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1254/2546] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:16:01.990107 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1edede-0736-4f05-b4f8-ca0e88504850 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:01.991527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0e1edede-0736-4f05-b4f8-ca0e88504850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:02.007171 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1edede-0736-4f05-b4f8-ca0e88504850 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:02.014339 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed976650-d7be-4ecc-a146-7d8eba024cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:02.015160 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed976650-d7be-4ecc-a146-7d8eba024cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:02.015539 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed976650-d7be-4ecc-a146-7d8eba024cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:02.015904 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed976650-d7be-4ecc-a146-7d8eba024cf3 admin admin] PATH_INFO: `/v3/projects/cfaae54fcadd43fa8a1f3f7e224519e8` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:02.017281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed976650-d7be-4ecc-a146-7d8eba024cf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfaae54fcadd43fa8a1f3f7e224519e8)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:02.019550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed976650-d7be-4ecc-a146-7d8eba024cf3 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:02.068885 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ed976650-d7be-4ecc-a146-7d8eba024cf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfaae54fcadd43fa8a1f3f7e224519e8', 'request_id': 'req-ed976650-d7be-4ecc-a146-7d8eba024cf3'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:02.076826 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ed976650-d7be-4ecc-a146-7d8eba024cf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfaae54fcadd43fa8a1f3f7e224519e8', 'request_id': 'req-ed976650-d7be-4ecc-a146-7d8eba024cf3'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:02.082793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ed976650-d7be-4ecc-a146-7d8eba024cf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfaae54fcadd43fa8a1f3f7e224519e8', 'request_id': 'req-ed976650-d7be-4ecc-a146-7d8eba024cf3'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:02.084786 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:02.097968 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ed976650-d7be-4ecc-a146-7d8eba024cf3 admin admin] The token cache is being invalidate because project cfaae54fcadd43fa8a1f3f7e224519e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:02.098266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ed976650-d7be-4ecc-a146-7d8eba024cf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:02.098935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-ed976650-d7be-4ecc-a146-7d8eba024cf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:02.100466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1293/2547] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:01 2022] DELETE /identity/v3/projects/cfaae54fcadd43fa8a1f3f7e224519e8 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:02.104537 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374689a2-d324-46fe-a25f-08516ba843a8 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:02.114667 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374689a2-d324-46fe-a25f-08516ba843a8 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:02.135500 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:02.174557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1f5700-9f88-4eb0-bf9a-50a3b77f373b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:02.175743 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f5700-9f88-4eb0-bf9a-50a3b77f373b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:02.176136 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f5700-9f88-4eb0-bf9a-50a3b77f373b admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:02.176418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1f5700-9f88-4eb0-bf9a-50a3b77f373b admin admin] PATH_INFO: `/v3/users/41f92c5167794d849247c0d6e436cde5` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:02.177577 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f5700-9f88-4eb0-bf9a-50a3b77f373b admin admin] RBAC: Authorizing `identity:delete_user(user_id=41f92c5167794d849247c0d6e436cde5)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:02.179553 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1f5700-9f88-4eb0-bf9a-50a3b77f373b admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:02.229851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-8c1f5700-9f88-4eb0-bf9a-50a3b77f373b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41f92c5167794d849247c0d6e436cde5'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:02.230853 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-8c1f5700-9f88-4eb0-bf9a-50a3b77f373b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41f92c5167794d849247c0d6e436cde5'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:02.243943 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-8c1f5700-9f88-4eb0-bf9a-50a3b77f373b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41f92c5167794d849247c0d6e436cde5'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:02.250892 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-8c1f5700-9f88-4eb0-bf9a-50a3b77f373b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41f92c5167794d849247c0d6e436cde5'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:02.251528 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-8c1f5700-9f88-4eb0-bf9a-50a3b77f373b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41f92c5167794d849247c0d6e436cde5'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:02.264768 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1255/2548] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:02 2022] DELETE /identity/v3/users/41f92c5167794d849247c0d6e436cde5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:02.577092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed976650-d7be-4ecc-a146-7d8eba024cf3 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:02.579787 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-ed976650-d7be-4ecc-a146-7d8eba024cf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:02.602498 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed976650-d7be-4ecc-a146-7d8eba024cf3 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:02.669612 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152494c-23f4-424c-971e-af3484a48248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:02.670263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152494c-23f4-424c-971e-af3484a48248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:02.670542 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152494c-23f4-424c-971e-af3484a48248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:02.670814 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152494c-23f4-424c-971e-af3484a48248 admin admin] PATH_INFO: `/v3/projects/c6fc792d15354a7aa22f9a3fedd29951` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:02.671996 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152494c-23f4-424c-971e-af3484a48248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6fc792d15354a7aa22f9a3fedd29951)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:02.673936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152494c-23f4-424c-971e-af3484a48248 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:02.737220 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-6152494c-23f4-424c-971e-af3484a48248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6fc792d15354a7aa22f9a3fedd29951', 'request_id': 'req-6152494c-23f4-424c-971e-af3484a48248'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:02.746076 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-6152494c-23f4-424c-971e-af3484a48248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6fc792d15354a7aa22f9a3fedd29951', 'request_id': 'req-6152494c-23f4-424c-971e-af3484a48248'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:02.756746 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-6152494c-23f4-424c-971e-af3484a48248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6fc792d15354a7aa22f9a3fedd29951', 'request_id': 'req-6152494c-23f4-424c-971e-af3484a48248'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:02.769387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-6152494c-23f4-424c-971e-af3484a48248 admin admin] The token cache is being invalidate because project c6fc792d15354a7aa22f9a3fedd29951 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:02.770189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-6152494c-23f4-424c-971e-af3484a48248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:02.770858 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-6152494c-23f4-424c-971e-af3484a48248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:02.776005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1294/2549] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:02 2022] DELETE /identity/v3/projects/c6fc792d15354a7aa22f9a3fedd29951 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:02.778365 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8512d00-affc-4890-a442-3654a112c791 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:02.778819 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8512d00-affc-4890-a442-3654a112c791 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:02.778957 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8512d00-affc-4890-a442-3654a112c791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:02.815223 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-e8512d00-affc-4890-a442-3654a112c791 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:02.821796 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-e8512d00-affc-4890-a442-3654a112c791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:02.910570 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1256/2550] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:16:02.918727 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d10c6-bf05-4400-a87d-d844b7a58cf8 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:02.919043 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d10c6-bf05-4400-a87d-d844b7a58cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:02.919417 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d10c6-bf05-4400-a87d-d844b7a58cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:02.956621 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-584d10c6-bf05-4400-a87d-d844b7a58cf8 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:02.958920 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-584d10c6-bf05-4400-a87d-d844b7a58cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:02.972677 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1295/2551] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:02.976937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8512d00-affc-4890-a442-3654a112c791 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:02.990278 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8512d00-affc-4890-a442-3654a112c791 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:02.997092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeab66a-3a39-45c8-8578-34ed7250be70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:02.997760 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeab66a-3a39-45c8-8578-34ed7250be70 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:02.998080 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeab66a-3a39-45c8-8578-34ed7250be70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:02.998582 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeab66a-3a39-45c8-8578-34ed7250be70 admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:02.999596 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeab66a-3a39-45c8-8578-34ed7250be70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:16:03.000143 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeab66a-3a39-45c8-8578-34ed7250be70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:03.002671 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeab66a-3a39-45c8-8578-34ed7250be70 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:03.022664 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1257/2552] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:03.024034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d10c6-bf05-4400-a87d-d844b7a58cf8 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:03.034897 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d10c6-bf05-4400-a87d-d844b7a58cf8 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:03.039889 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94036ce1-1121-4c0d-aa66-ac3932cd8d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:03.040508 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94036ce1-1121-4c0d-aa66-ac3932cd8d8c admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:03.040792 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94036ce1-1121-4c0d-aa66-ac3932cd8d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:03.041069 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94036ce1-1121-4c0d-aa66-ac3932cd8d8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:03.042694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94036ce1-1121-4c0d-aa66-ac3932cd8d8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:03.044697 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94036ce1-1121-4c0d-aa66-ac3932cd8d8c admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:03.083631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1296/2553] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:16:03 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:16:03.087841 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeab66a-3a39-45c8-8578-34ed7250be70 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:03.104732 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeab66a-3a39-45c8-8578-34ed7250be70 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:03.138553 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:03.184875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9d905-7cd2-4c8a-98cc-578aa6c09295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:03.185521 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9d905-7cd2-4c8a-98cc-578aa6c09295 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:03.185807 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9d905-7cd2-4c8a-98cc-578aa6c09295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:03.186101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc9d905-7cd2-4c8a-98cc-578aa6c09295 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:03.186743 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc9d905-7cd2-4c8a-98cc-578aa6c09295 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:03.188807 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc9d905-7cd2-4c8a-98cc-578aa6c09295 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:03.211213 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1258/2554] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:16:03 2022] POST /identity/v3/users => generated 292 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:16:03.215211 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94036ce1-1121-4c0d-aa66-ac3932cd8d8c admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:03.226028 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94036ce1-1121-4c0d-aa66-ac3932cd8d8c admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:03.230754 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adc9f0-f750-41a1-8a08-c9335f9c9948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:03.231411 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adc9f0-f750-41a1-8a08-c9335f9c9948 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:03.231719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adc9f0-f750-41a1-8a08-c9335f9c9948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:03.231997 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adc9f0-f750-41a1-8a08-c9335f9c9948 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:03.232741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adc9f0-f750-41a1-8a08-c9335f9c9948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:03.234403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adc9f0-f750-41a1-8a08-c9335f9c9948 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:03.245441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1297/2555] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:03.249507 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9d905-7cd2-4c8a-98cc-578aa6c09295 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:03.261443 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc9d905-7cd2-4c8a-98cc-578aa6c09295 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:03.266283 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0100553-dcea-4542-bb98-c7332690ab2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:03.266891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0100553-dcea-4542-bb98-c7332690ab2a admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:03.267204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0100553-dcea-4542-bb98-c7332690ab2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:03.267482 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0100553-dcea-4542-bb98-c7332690ab2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:03.268123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0100553-dcea-4542-bb98-c7332690ab2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:03.269780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0100553-dcea-4542-bb98-c7332690ab2a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:03.283194 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1259/2556] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:16:03.286151 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adc9f0-f750-41a1-8a08-c9335f9c9948 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:03.298253 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adc9f0-f750-41a1-8a08-c9335f9c9948 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:03.304986 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378a1b4-fb9d-40e2-ba12-131b64096a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:03.305662 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378a1b4-fb9d-40e2-ba12-131b64096a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:03.305944 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378a1b4-fb9d-40e2-ba12-131b64096a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:03.306215 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378a1b4-fb9d-40e2-ba12-131b64096a2b admin admin] PATH_INFO: `/v3/projects/de94fa3871d740e8add920033f8844df/users/690d78909aa540419c08730952431849/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:03.319706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0100553-dcea-4542-bb98-c7332690ab2a admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:03.321509 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-e0100553-dcea-4542-bb98-c7332690ab2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:03.326872 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f378a1b4-fb9d-40e2-ba12-131b64096a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=de94fa3871d740e8add920033f8844df, user_id=690d78909aa540419c08730952431849, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:03.329965 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f378a1b4-fb9d-40e2-ba12-131b64096a2b admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:03.337077 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0100553-dcea-4542-bb98-c7332690ab2a admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:03.342778 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a388897-e6bf-45d0-935c-bd897150b9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:03.343097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a388897-e6bf-45d0-935c-bd897150b9bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:03.343425 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a388897-e6bf-45d0-935c-bd897150b9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:03.343700 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a388897-e6bf-45d0-935c-bd897150b9bb admin admin] PATH_INFO: `/v3/users/e8b5ce2249a84c01ab57c03b656b2949` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:03.344774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1298/2557] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:03 2022] PUT /identity/v3/projects/de94fa3871d740e8add920033f8844df/users/690d78909aa540419c08730952431849/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:03.344937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a388897-e6bf-45d0-935c-bd897150b9bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8b5ce2249a84c01ab57c03b656b2949)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:03.347684 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a388897-e6bf-45d0-935c-bd897150b9bb admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:03.355579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d9626-861c-495f-81f7-3fbdd38f00bc None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:03.355864 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d9626-861c-495f-81f7-3fbdd38f00bc None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:03.356132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d9626-861c-495f-81f7-3fbdd38f00bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:03.384845 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-355d9626-861c-495f-81f7-3fbdd38f00bc None None] MFA Rules not processed for user `690d78909aa540419c08730952431849`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:03.386314 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-355d9626-861c-495f-81f7-3fbdd38f00bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:03.398649 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:03.401184 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-3a388897-e6bf-45d0-935c-bd897150b9bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8b5ce2249a84c01ab57c03b656b2949'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:03.402054 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-3a388897-e6bf-45d0-935c-bd897150b9bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8b5ce2249a84c01ab57c03b656b2949'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:03.415524 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-3a388897-e6bf-45d0-935c-bd897150b9bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8b5ce2249a84c01ab57c03b656b2949'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:03.422403 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-3a388897-e6bf-45d0-935c-bd897150b9bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8b5ce2249a84c01ab57c03b656b2949'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:03.423137 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-3a388897-e6bf-45d0-935c-bd897150b9bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8b5ce2249a84c01ab57c03b656b2949'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:03.436523 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1260/2558] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:03 2022] DELETE /identity/v3/users/e8b5ce2249a84c01ab57c03b656b2949 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:16:03.457574 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1299/2559] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:03.468221 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a388897-e6bf-45d0-935c-bd897150b9bb admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:03.469644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-3a388897-e6bf-45d0-935c-bd897150b9bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:03.532892 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a388897-e6bf-45d0-935c-bd897150b9bb admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:03.536548 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d92fa4-86a6-4fe6-9458-8c01a47f60ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:03.537155 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d92fa4-86a6-4fe6-9458-8c01a47f60ab service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:03.537430 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d92fa4-86a6-4fe6-9458-8c01a47f60ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:03.537703 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d92fa4-86a6-4fe6-9458-8c01a47f60ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:03.572559 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:03.599564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d92fa4-86a6-4fe6-9458-8c01a47f60ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:03.601001 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d92fa4-86a6-4fe6-9458-8c01a47f60ab service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:03.610496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1261/2560] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:16:03.612685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d9626-861c-495f-81f7-3fbdd38f00bc None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:03.614730 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-355d9626-861c-495f-81f7-3fbdd38f00bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:03.689438 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d9626-861c-495f-81f7-3fbdd38f00bc None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:03.746822 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cf158d-e8d3-472f-8df3-72c42a24d9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:03.747485 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf158d-e8d3-472f-8df3-72c42a24d9ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:03.747763 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf158d-e8d3-472f-8df3-72c42a24d9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:03.748037 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cf158d-e8d3-472f-8df3-72c42a24d9ec admin admin] PATH_INFO: `/v3/projects/3450e9ca29fb44e4808fa52de2f88470` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:03.749164 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf158d-e8d3-472f-8df3-72c42a24d9ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=3450e9ca29fb44e4808fa52de2f88470)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:03.751116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cf158d-e8d3-472f-8df3-72c42a24d9ec admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:03.812549 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-44cf158d-e8d3-472f-8df3-72c42a24d9ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3450e9ca29fb44e4808fa52de2f88470', 'request_id': 'req-44cf158d-e8d3-472f-8df3-72c42a24d9ec'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:03.820689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-44cf158d-e8d3-472f-8df3-72c42a24d9ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3450e9ca29fb44e4808fa52de2f88470', 'request_id': 'req-44cf158d-e8d3-472f-8df3-72c42a24d9ec'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:03.826139 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:03.829134 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-44cf158d-e8d3-472f-8df3-72c42a24d9ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3450e9ca29fb44e4808fa52de2f88470', 'request_id': 'req-44cf158d-e8d3-472f-8df3-72c42a24d9ec'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:03.845338 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-44cf158d-e8d3-472f-8df3-72c42a24d9ec admin admin] The token cache is being invalidate because project 3450e9ca29fb44e4808fa52de2f88470 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:03.845655 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-44cf158d-e8d3-472f-8df3-72c42a24d9ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:03.847678 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-44cf158d-e8d3-472f-8df3-72c42a24d9ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:03.851169 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1300/2561] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:03 2022] DELETE /identity/v3/projects/3450e9ca29fb44e4808fa52de2f88470 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:03.856354 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724512c2-4422-4cfd-be0c-add9af4a72c6 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:03.857195 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724512c2-4422-4cfd-be0c-add9af4a72c6 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:03.857463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724512c2-4422-4cfd-be0c-add9af4a72c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:03.881909 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-724512c2-4422-4cfd-be0c-add9af4a72c6 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:03.883996 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-724512c2-4422-4cfd-be0c-add9af4a72c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:03.957473 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1262/2562] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:03.966856 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddba6ac-2892-4a0b-bb3b-a78b5b37d404 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:03.967174 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddba6ac-2892-4a0b-bb3b-a78b5b37d404 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:03.967773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddba6ac-2892-4a0b-bb3b-a78b5b37d404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:03.994754 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-1ddba6ac-2892-4a0b-bb3b-a78b5b37d404 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:03.996087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-1ddba6ac-2892-4a0b-bb3b-a78b5b37d404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:04.004617 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1301/2563] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:04.009229 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724512c2-4422-4cfd-be0c-add9af4a72c6 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:04.022348 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724512c2-4422-4cfd-be0c-add9af4a72c6 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:04.027835 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312f487-06d8-470d-826f-271bef7f8863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:04.028514 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312f487-06d8-470d-826f-271bef7f8863 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:04.028789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312f487-06d8-470d-826f-271bef7f8863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:04.029060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312f487-06d8-470d-826f-271bef7f8863 admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:04.029834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312f487-06d8-470d-826f-271bef7f8863 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:16:04.030189 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312f487-06d8-470d-826f-271bef7f8863 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:04.031926 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312f487-06d8-470d-826f-271bef7f8863 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:04.045449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1263/2564] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:16:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:04.049997 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddba6ac-2892-4a0b-bb3b-a78b5b37d404 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:04.059230 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddba6ac-2892-4a0b-bb3b-a78b5b37d404 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:04.063898 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30230d-a500-435c-bb4a-c7dbe726ad8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:04.064493 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30230d-a500-435c-bb4a-c7dbe726ad8e admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:04.064772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30230d-a500-435c-bb4a-c7dbe726ad8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:04.065044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c30230d-a500-435c-bb4a-c7dbe726ad8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:04.065730 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30230d-a500-435c-bb4a-c7dbe726ad8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:04.067699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c30230d-a500-435c-bb4a-c7dbe726ad8e admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:04.097967 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1302/2565] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:16:04 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:04.102377 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312f487-06d8-470d-826f-271bef7f8863 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:04.113583 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312f487-06d8-470d-826f-271bef7f8863 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:04.173698 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c6140-b14b-4c91-b365-0c25a8fe2d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:04.174249 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c6140-b14b-4c91-b365-0c25a8fe2d98 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:04.174612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c6140-b14b-4c91-b365-0c25a8fe2d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:04.175358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c6140-b14b-4c91-b365-0c25a8fe2d98 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:04.176022 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c6140-b14b-4c91-b365-0c25a8fe2d98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:04.177986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c6140-b14b-4c91-b365-0c25a8fe2d98 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:04.203449 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30230d-a500-435c-bb4a-c7dbe726ad8e admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:04.213805 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1264/2566] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:16:04 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:04.218118 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c30230d-a500-435c-bb4a-c7dbe726ad8e admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:04.223128 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7360f2-1900-410f-b98a-68ce6b7ce08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:04.223752 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7360f2-1900-410f-b98a-68ce6b7ce08d admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:04.224022 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7360f2-1900-410f-b98a-68ce6b7ce08d admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:04.225756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7360f2-1900-410f-b98a-68ce6b7ce08d admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:04.226411 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7360f2-1900-410f-b98a-68ce6b7ce08d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:04.228403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7360f2-1900-410f-b98a-68ce6b7ce08d admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:04.241544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1303/2567] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:04.242821 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c6140-b14b-4c91-b365-0c25a8fe2d98 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:04.254733 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c6140-b14b-4c91-b365-0c25a8fe2d98 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:04.260022 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae73c8-bde9-438c-ab3b-e97b92593ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:04.260634 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae73c8-bde9-438c-ab3b-e97b92593ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:04.260913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae73c8-bde9-438c-ab3b-e97b92593ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:04.261184 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae73c8-bde9-438c-ab3b-e97b92593ad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:04.261829 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae73c8-bde9-438c-ab3b-e97b92593ad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:04.263442 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae73c8-bde9-438c-ab3b-e97b92593ad7 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:04.274837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1265/2568] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:04.279251 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7360f2-1900-410f-b98a-68ce6b7ce08d admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:04.290023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7360f2-1900-410f-b98a-68ce6b7ce08d admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:04.294769 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820dfae5-5b56-4c73-864f-35fd999cc9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:04.295522 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820dfae5-5b56-4c73-864f-35fd999cc9b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:04.295748 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820dfae5-5b56-4c73-864f-35fd999cc9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:04.296020 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820dfae5-5b56-4c73-864f-35fd999cc9b9 admin admin] PATH_INFO: `/v3/projects/a15cc4d6f0904b17b8e80af9bee42f9d/users/0cd2dbb2c54449eeb2811cbb5a1e3929/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:04.311006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820dfae5-5b56-4c73-864f-35fd999cc9b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a15cc4d6f0904b17b8e80af9bee42f9d, user_id=0cd2dbb2c54449eeb2811cbb5a1e3929, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:04.313736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820dfae5-5b56-4c73-864f-35fd999cc9b9 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:04.325935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1304/2569] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:04 2022] PUT /identity/v3/projects/a15cc4d6f0904b17b8e80af9bee42f9d/users/0cd2dbb2c54449eeb2811cbb5a1e3929/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:04.336293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a10ae-b453-4357-9a3b-29d34ca471ab None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:04.336563 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a10ae-b453-4357-9a3b-29d34ca471ab None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:04.336839 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a10ae-b453-4357-9a3b-29d34ca471ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:04.366625 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-992a10ae-b453-4357-9a3b-29d34ca471ab None None] MFA Rules not processed for user `0cd2dbb2c54449eeb2811cbb5a1e3929`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:04.367939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-992a10ae-b453-4357-9a3b-29d34ca471ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:04.398585 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:04.432923 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1266/2570] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:04.444312 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820dfae5-5b56-4c73-864f-35fd999cc9b9 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:04.446022 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-820dfae5-5b56-4c73-864f-35fd999cc9b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:04.521525 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820dfae5-5b56-4c73-864f-35fd999cc9b9 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:04.526990 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e474bc-8c83-4872-9737-c84194131f02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:04.527623 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e474bc-8c83-4872-9737-c84194131f02 service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:04.527897 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e474bc-8c83-4872-9737-c84194131f02 service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:04.528163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e474bc-8c83-4872-9737-c84194131f02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:04.536393 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e474bc-8c83-4872-9737-c84194131f02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:04.538101 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e474bc-8c83-4872-9737-c84194131f02 service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:04.547092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1305/2571] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:16:04.766845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a10ae-b453-4357-9a3b-29d34ca471ab None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:04.835285 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a10ae-b453-4357-9a3b-29d34ca471ab None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:04.839930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582b1a0-890b-460b-835e-3221870a66da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:04.840591 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582b1a0-890b-460b-835e-3221870a66da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:04.840871 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582b1a0-890b-460b-835e-3221870a66da admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:04.841148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582b1a0-890b-460b-835e-3221870a66da admin admin] PATH_INFO: `/v3/users/0cd2dbb2c54449eeb2811cbb5a1e3929` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:04.842279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4582b1a0-890b-460b-835e-3221870a66da admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cd2dbb2c54449eeb2811cbb5a1e3929)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:04.844231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4582b1a0-890b-460b-835e-3221870a66da admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:04.896245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4582b1a0-890b-460b-835e-3221870a66da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cd2dbb2c54449eeb2811cbb5a1e3929'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:04.897061 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4582b1a0-890b-460b-835e-3221870a66da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cd2dbb2c54449eeb2811cbb5a1e3929'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:04.910784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4582b1a0-890b-460b-835e-3221870a66da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cd2dbb2c54449eeb2811cbb5a1e3929'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:04.917425 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4582b1a0-890b-460b-835e-3221870a66da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cd2dbb2c54449eeb2811cbb5a1e3929'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:04.918056 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4582b1a0-890b-460b-835e-3221870a66da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cd2dbb2c54449eeb2811cbb5a1e3929'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:04.930782 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1267/2572] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:04 2022] DELETE /identity/v3/users/0cd2dbb2c54449eeb2811cbb5a1e3929 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:04.935248 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1417b-12ac-4b19-a953-8e378d7688a0 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:04.935700 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1417b-12ac-4b19-a953-8e378d7688a0 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:04.935827 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1417b-12ac-4b19-a953-8e378d7688a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:04.957671 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-afe1417b-12ac-4b19-a953-8e378d7688a0 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:04.958881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-afe1417b-12ac-4b19-a953-8e378d7688a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:05.021568 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1306/2573] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:05.033237 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582b1a0-890b-460b-835e-3221870a66da admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:05.033746 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-4582b1a0-890b-460b-835e-3221870a66da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:05.094857 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582b1a0-890b-460b-835e-3221870a66da admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:05.099879 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ce0e99-217e-4156-8932-80a1c25c5c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:05.102180 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce0e99-217e-4156-8932-80a1c25c5c88 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:05.102180 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce0e99-217e-4156-8932-80a1c25c5c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:05.102180 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce0e99-217e-4156-8932-80a1c25c5c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:05.112519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:05.118456 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce0e99-217e-4156-8932-80a1c25c5c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:05.120978 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ce0e99-217e-4156-8932-80a1c25c5c88 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:05.132143 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1268/2574] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:16:05.430920 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1417b-12ac-4b19-a953-8e378d7688a0 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:05.430920 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-afe1417b-12ac-4b19-a953-8e378d7688a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:05.445331 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1417b-12ac-4b19-a953-8e378d7688a0 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:05.449986 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a00ae8-f240-457f-83ad-17b1a611df64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:05.450627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a00ae8-f240-457f-83ad-17b1a611df64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:05.450906 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a00ae8-f240-457f-83ad-17b1a611df64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:05.451211 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a00ae8-f240-457f-83ad-17b1a611df64 admin admin] PATH_INFO: `/v3/projects/a15cc4d6f0904b17b8e80af9bee42f9d` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:05.452318 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a00ae8-f240-457f-83ad-17b1a611df64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a15cc4d6f0904b17b8e80af9bee42f9d)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:05.454285 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a00ae8-f240-457f-83ad-17b1a611df64 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:05.461634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:05.502800 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-81a00ae8-f240-457f-83ad-17b1a611df64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a15cc4d6f0904b17b8e80af9bee42f9d', 'request_id': 'req-81a00ae8-f240-457f-83ad-17b1a611df64'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:05.510613 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-81a00ae8-f240-457f-83ad-17b1a611df64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a15cc4d6f0904b17b8e80af9bee42f9d', 'request_id': 'req-81a00ae8-f240-457f-83ad-17b1a611df64'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:05.517733 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-81a00ae8-f240-457f-83ad-17b1a611df64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a15cc4d6f0904b17b8e80af9bee42f9d', 'request_id': 'req-81a00ae8-f240-457f-83ad-17b1a611df64'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:05.532069 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-81a00ae8-f240-457f-83ad-17b1a611df64 admin admin] The token cache is being invalidate because project a15cc4d6f0904b17b8e80af9bee42f9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:05.532357 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-81a00ae8-f240-457f-83ad-17b1a611df64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:05.533042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-81a00ae8-f240-457f-83ad-17b1a611df64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:05.534547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1307/2575] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:05 2022] DELETE /identity/v3/projects/a15cc4d6f0904b17b8e80af9bee42f9d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:05.539382 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54408edb-9455-434b-9a79-b921fd781d9d None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:05.539664 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54408edb-9455-434b-9a79-b921fd781d9d None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:05.539936 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54408edb-9455-434b-9a79-b921fd781d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:05.561342 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-54408edb-9455-434b-9a79-b921fd781d9d None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:05.563486 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-54408edb-9455-434b-9a79-b921fd781d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:05.629934 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1269/2576] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:05.639120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291bf55-6a7a-44b8-8271-8e4e29a95d33 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:05.639473 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291bf55-6a7a-44b8-8271-8e4e29a95d33 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:05.639756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8291bf55-6a7a-44b8-8271-8e4e29a95d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:05.659446 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-8291bf55-6a7a-44b8-8271-8e4e29a95d33 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:05.660754 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8291bf55-6a7a-44b8-8271-8e4e29a95d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:05.670977 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1308/2577] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:05.676006 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54408edb-9455-434b-9a79-b921fd781d9d None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:05.685737 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54408edb-9455-434b-9a79-b921fd781d9d None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:05.690284 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace65a1-1537-4dee-b4c8-4a24a799ae67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:05.691470 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace65a1-1537-4dee-b4c8-4a24a799ae67 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:05.691750 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace65a1-1537-4dee-b4c8-4a24a799ae67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:05.692022 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace65a1-1537-4dee-b4c8-4a24a799ae67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:05.692688 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace65a1-1537-4dee-b4c8-4a24a799ae67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:16:05.692992 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace65a1-1537-4dee-b4c8-4a24a799ae67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:05.694571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ace65a1-1537-4dee-b4c8-4a24a799ae67 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:05.706121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1270/2578] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:05.710626 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291bf55-6a7a-44b8-8271-8e4e29a95d33 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:05.719728 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8291bf55-6a7a-44b8-8271-8e4e29a95d33 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:05.725522 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87471d6e-815f-482e-8ea8-43d0c056aeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:05.726142 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87471d6e-815f-482e-8ea8-43d0c056aeac admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:05.726461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87471d6e-815f-482e-8ea8-43d0c056aeac admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:05.726726 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87471d6e-815f-482e-8ea8-43d0c056aeac admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:05.727386 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87471d6e-815f-482e-8ea8-43d0c056aeac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:05.729457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87471d6e-815f-482e-8ea8-43d0c056aeac admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:05.758449 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1309/2579] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:16:05 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:05.762563 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace65a1-1537-4dee-b4c8-4a24a799ae67 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:05.772693 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace65a1-1537-4dee-b4c8-4a24a799ae67 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:05.833981 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ecfd2-4df4-431a-a3ac-12e6385653d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:05.834578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ecfd2-4df4-431a-a3ac-12e6385653d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:05.834883 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ecfd2-4df4-431a-a3ac-12e6385653d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:05.835271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759ecfd2-4df4-431a-a3ac-12e6385653d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:05.835973 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ecfd2-4df4-431a-a3ac-12e6385653d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:05.838181 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759ecfd2-4df4-431a-a3ac-12e6385653d5 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:05.860359 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1271/2580] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:16:05 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:05.864828 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87471d6e-815f-482e-8ea8-43d0c056aeac admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:05.879214 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87471d6e-815f-482e-8ea8-43d0c056aeac admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:05.884364 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fabfc-9438-4f85-aa26-f4b035bbdaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:05.884901 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fabfc-9438-4f85-aa26-f4b035bbdaff admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:05.885177 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fabfc-9438-4f85-aa26-f4b035bbdaff admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:05.885445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45fabfc-9438-4f85-aa26-f4b035bbdaff admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:05.886075 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fabfc-9438-4f85-aa26-f4b035bbdaff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:05.887699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45fabfc-9438-4f85-aa26-f4b035bbdaff admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:05.898561 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1310/2581] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:05.902662 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ecfd2-4df4-431a-a3ac-12e6385653d5 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:05.912875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759ecfd2-4df4-431a-a3ac-12e6385653d5 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:05.917488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52915820-b725-4dbe-82c2-33196215cdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:05.918116 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52915820-b725-4dbe-82c2-33196215cdf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:05.918395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52915820-b725-4dbe-82c2-33196215cdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:05.918670 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52915820-b725-4dbe-82c2-33196215cdf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:05.919332 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52915820-b725-4dbe-82c2-33196215cdf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:05.920915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52915820-b725-4dbe-82c2-33196215cdf9 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:05.933776 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1272/2582] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:05.938233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fabfc-9438-4f85-aa26-f4b035bbdaff admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:05.948858 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45fabfc-9438-4f85-aa26-f4b035bbdaff admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:05.953351 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74dd7f2-578c-4881-a30e-3d573b6117db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:05.953986 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74dd7f2-578c-4881-a30e-3d573b6117db admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:05.954262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74dd7f2-578c-4881-a30e-3d573b6117db admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:05.954532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74dd7f2-578c-4881-a30e-3d573b6117db admin admin] PATH_INFO: `/v3/projects/fe0b9498115b4068b5ab47489bcd603c/users/28f8692ecdd54e7fab17d7fa7d7414e9/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:05.970552 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74dd7f2-578c-4881-a30e-3d573b6117db admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe0b9498115b4068b5ab47489bcd603c, user_id=28f8692ecdd54e7fab17d7fa7d7414e9, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:05.973345 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74dd7f2-578c-4881-a30e-3d573b6117db admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:05.985857 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1311/2583] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:05 2022] PUT /identity/v3/projects/fe0b9498115b4068b5ab47489bcd603c/users/28f8692ecdd54e7fab17d7fa7d7414e9/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:05.996744 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4ed79-0025-4119-ab86-6b207f64d109 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:05.997295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4ed79-0025-4119-ab86-6b207f64d109 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:05.997736 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4ed79-0025-4119-ab86-6b207f64d109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:06.025886 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-8aa4ed79-0025-4119-ab86-6b207f64d109 None None] MFA Rules not processed for user `28f8692ecdd54e7fab17d7fa7d7414e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:06.027109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-8aa4ed79-0025-4119-ab86-6b207f64d109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:06.073115 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:06.092147 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1273/2584] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:06.096655 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74dd7f2-578c-4881-a30e-3d573b6117db admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:06.159951 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74dd7f2-578c-4881-a30e-3d573b6117db admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:06.164571 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8b048-0d0d-4a5a-bea5-0fe17a583a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:06.165161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8b048-0d0d-4a5a-bea5-0fe17a583a30 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:06.165437 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8b048-0d0d-4a5a-bea5-0fe17a583a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:06.165703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8b048-0d0d-4a5a-bea5-0fe17a583a30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:06.166311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8b048-0d0d-4a5a-bea5-0fe17a583a30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:06.168218 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef8b048-0d0d-4a5a-bea5-0fe17a583a30 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:06.195104 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1312/2585] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:16:06 2022] POST /identity/v3/projects => generated 363 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:06.199384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4ed79-0025-4119-ab86-6b207f64d109 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:06.211437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4ed79-0025-4119-ab86-6b207f64d109 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:06.267435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3baf36-21e3-43d7-87d4-bc9ae73f4bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:06.268032 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3baf36-21e3-43d7-87d4-bc9ae73f4bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:06.268306 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3baf36-21e3-43d7-87d4-bc9ae73f4bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:06.268603 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3baf36-21e3-43d7-87d4-bc9ae73f4bb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:06.269227 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3baf36-21e3-43d7-87d4-bc9ae73f4bb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:06.271124 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3baf36-21e3-43d7-87d4-bc9ae73f4bb8 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:06.291363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1274/2586] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:16:06 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:06.295564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8b048-0d0d-4a5a-bea5-0fe17a583a30 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:06.306650 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8b048-0d0d-4a5a-bea5-0fe17a583a30 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:06.311431 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a89ec5-3aa3-4056-a65d-eba37b0c16de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:06.312037 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a89ec5-3aa3-4056-a65d-eba37b0c16de admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:06.312313 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a89ec5-3aa3-4056-a65d-eba37b0c16de admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:06.312579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a89ec5-3aa3-4056-a65d-eba37b0c16de admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:06.313214 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a89ec5-3aa3-4056-a65d-eba37b0c16de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:06.314831 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a89ec5-3aa3-4056-a65d-eba37b0c16de admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:06.325829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1313/2587] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:06.330038 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3baf36-21e3-43d7-87d4-bc9ae73f4bb8 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:06.340778 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3baf36-21e3-43d7-87d4-bc9ae73f4bb8 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:06.345506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4722ed7a-e075-467c-b089-73af4a6a0306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:06.346164 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722ed7a-e075-467c-b089-73af4a6a0306 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:06.346442 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722ed7a-e075-467c-b089-73af4a6a0306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:06.346768 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4722ed7a-e075-467c-b089-73af4a6a0306 admin admin] PATH_INFO: `/v3/projects/61e270874cb54f0ab19153d4171fd2f8/users/9ae700f4e75545e2bc0fd5f4d0c6886d/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:06.364565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4722ed7a-e075-467c-b089-73af4a6a0306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61e270874cb54f0ab19153d4171fd2f8, user_id=9ae700f4e75545e2bc0fd5f4d0c6886d, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:06.367175 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4722ed7a-e075-467c-b089-73af4a6a0306 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:06.384630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1275/2588] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:06 2022] PUT /identity/v3/projects/61e270874cb54f0ab19153d4171fd2f8/users/9ae700f4e75545e2bc0fd5f4d0c6886d/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:06.388823 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a89ec5-3aa3-4056-a65d-eba37b0c16de admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:06.395047 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:06.399258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a89ec5-3aa3-4056-a65d-eba37b0c16de admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:06.460921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731eeb6d-84ad-4d48-9cea-f0a97e8c3473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:06.461441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731eeb6d-84ad-4d48-9cea-f0a97e8c3473 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:06.461714 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731eeb6d-84ad-4d48-9cea-f0a97e8c3473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:06.461980 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731eeb6d-84ad-4d48-9cea-f0a97e8c3473 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:06.462641 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731eeb6d-84ad-4d48-9cea-f0a97e8c3473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:06.464246 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731eeb6d-84ad-4d48-9cea-f0a97e8c3473 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:06.474086 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1314/2589] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:06.478246 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4722ed7a-e075-467c-b089-73af4a6a0306 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:06.488919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4722ed7a-e075-467c-b089-73af4a6a0306 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:06.493459 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e6328-be46-463c-8410-b11fca85f632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:06.494096 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e6328-be46-463c-8410-b11fca85f632 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:06.494371 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e6328-be46-463c-8410-b11fca85f632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:06.494640 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e6328-be46-463c-8410-b11fca85f632 admin admin] PATH_INFO: `/v3/projects/61e270874cb54f0ab19153d4171fd2f8/users/9ae700f4e75545e2bc0fd5f4d0c6886d/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:06.496581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e6328-be46-463c-8410-b11fca85f632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61e270874cb54f0ab19153d4171fd2f8, user_id=9ae700f4e75545e2bc0fd5f4d0c6886d, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:06.499205 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e6328-be46-463c-8410-b11fca85f632 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:06.510312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1276/2590] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:06 2022] PUT /identity/v3/projects/61e270874cb54f0ab19153d4171fd2f8/users/9ae700f4e75545e2bc0fd5f4d0c6886d/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:06.520983 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e6db9-9236-4621-bdda-48f23b924a1d None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:06.521263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e6db9-9236-4621-bdda-48f23b924a1d None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:06.521535 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e6db9-9236-4621-bdda-48f23b924a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:06.548864 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-d01e6db9-9236-4621-bdda-48f23b924a1d None None] MFA Rules not processed for user `9ae700f4e75545e2bc0fd5f4d0c6886d`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:06.550549 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-d01e6db9-9236-4621-bdda-48f23b924a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:06.618181 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1315/2591] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:06.629994 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e6328-be46-463c-8410-b11fca85f632 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:06.631805 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-678e6328-be46-463c-8410-b11fca85f632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:06.688971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e6328-be46-463c-8410-b11fca85f632 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:06.693281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcfa623-1f9f-4491-a6e9-2aad8825da56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:06.693885 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfa623-1f9f-4491-a6e9-2aad8825da56 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:06.694167 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfa623-1f9f-4491-a6e9-2aad8825da56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:06.694436 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfa623-1f9f-4491-a6e9-2aad8825da56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:06.703212 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcfa623-1f9f-4491-a6e9-2aad8825da56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:06.704630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcfa623-1f9f-4491-a6e9-2aad8825da56 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:06.714249 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1277/2592] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:06 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:16:07.129890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e6db9-9236-4621-bdda-48f23b924a1d None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:07.202511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e6db9-9236-4621-bdda-48f23b924a1d None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:07.207819 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8dcade-f0ce-4aff-9884-c5cf972eceb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:07.208506 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dcade-f0ce-4aff-9884-c5cf972eceb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:07.208815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dcade-f0ce-4aff-9884-c5cf972eceb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:07.209066 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8dcade-f0ce-4aff-9884-c5cf972eceb0 admin admin] PATH_INFO: `/v3/users/28f8692ecdd54e7fab17d7fa7d7414e9` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:07.210175 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8dcade-f0ce-4aff-9884-c5cf972eceb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28f8692ecdd54e7fab17d7fa7d7414e9)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:07.212126 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8dcade-f0ce-4aff-9884-c5cf972eceb0 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:07.245428 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:07.263321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-8e8dcade-f0ce-4aff-9884-c5cf972eceb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28f8692ecdd54e7fab17d7fa7d7414e9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:07.270719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-8e8dcade-f0ce-4aff-9884-c5cf972eceb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28f8692ecdd54e7fab17d7fa7d7414e9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:07.278775 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-8e8dcade-f0ce-4aff-9884-c5cf972eceb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28f8692ecdd54e7fab17d7fa7d7414e9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:07.279421 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-8e8dcade-f0ce-4aff-9884-c5cf972eceb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28f8692ecdd54e7fab17d7fa7d7414e9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:07.280026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-8e8dcade-f0ce-4aff-9884-c5cf972eceb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28f8692ecdd54e7fab17d7fa7d7414e9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:07.297938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1316/2593] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:07 2022] DELETE /identity/v3/users/28f8692ecdd54e7fab17d7fa7d7414e9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:07.303113 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588dccc-fc70-473e-b01a-afe11440d184 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:07.303707 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588dccc-fc70-473e-b01a-afe11440d184 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:07.303707 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588dccc-fc70-473e-b01a-afe11440d184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:07.322316 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-2588dccc-fc70-473e-b01a-afe11440d184 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:07.323541 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-2588dccc-fc70-473e-b01a-afe11440d184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:07.365864 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:07.386254 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1278/2594] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:07.396711 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8dcade-f0ce-4aff-9884-c5cf972eceb0 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:07.398098 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8e8dcade-f0ce-4aff-9884-c5cf972eceb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:07.459661 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8dcade-f0ce-4aff-9884-c5cf972eceb0 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:07.463495 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88b645-ec39-4640-b144-0000ebad6b2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:07.464130 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88b645-ec39-4640-b144-0000ebad6b2b service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:07.464407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88b645-ec39-4640-b144-0000ebad6b2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:07.464679 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88b645-ec39-4640-b144-0000ebad6b2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:07.474360 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88b645-ec39-4640-b144-0000ebad6b2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:07.475829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88b645-ec39-4640-b144-0000ebad6b2b service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:07.485067 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1317/2595] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:16:07.753596 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2588dccc-fc70-473e-b01a-afe11440d184 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:07.755125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-2588dccc-fc70-473e-b01a-afe11440d184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:07.766918 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2588dccc-fc70-473e-b01a-afe11440d184 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:07.771890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c594ef3-f5ff-479f-8c0b-5fb42da80a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:07.772527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c594ef3-f5ff-479f-8c0b-5fb42da80a35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:07.772807 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c594ef3-f5ff-479f-8c0b-5fb42da80a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:07.773077 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c594ef3-f5ff-479f-8c0b-5fb42da80a35 admin admin] PATH_INFO: `/v3/projects/fe0b9498115b4068b5ab47489bcd603c` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:07.774202 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c594ef3-f5ff-479f-8c0b-5fb42da80a35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe0b9498115b4068b5ab47489bcd603c)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:07.776169 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c594ef3-f5ff-479f-8c0b-5fb42da80a35 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:07.825211 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-1c594ef3-f5ff-479f-8c0b-5fb42da80a35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe0b9498115b4068b5ab47489bcd603c', 'request_id': 'req-1c594ef3-f5ff-479f-8c0b-5fb42da80a35'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:07.831615 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-1c594ef3-f5ff-479f-8c0b-5fb42da80a35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe0b9498115b4068b5ab47489bcd603c', 'request_id': 'req-1c594ef3-f5ff-479f-8c0b-5fb42da80a35'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:07.837862 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-1c594ef3-f5ff-479f-8c0b-5fb42da80a35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe0b9498115b4068b5ab47489bcd603c', 'request_id': 'req-1c594ef3-f5ff-479f-8c0b-5fb42da80a35'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:07.850364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-1c594ef3-f5ff-479f-8c0b-5fb42da80a35 admin admin] The token cache is being invalidate because project fe0b9498115b4068b5ab47489bcd603c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:07.850659 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-1c594ef3-f5ff-479f-8c0b-5fb42da80a35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:07.851285 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-1c594ef3-f5ff-479f-8c0b-5fb42da80a35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:07.852750 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1279/2596] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:07 2022] DELETE /identity/v3/projects/fe0b9498115b4068b5ab47489bcd603c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:07.856863 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88b645-ec39-4640-b144-0000ebad6b2b service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:07.866962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88b645-ec39-4640-b144-0000ebad6b2b service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:07.929628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dfa377-a798-49f7-88a4-f694c2f784da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:07.930279 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfa377-a798-49f7-88a4-f694c2f784da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:07.930560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfa377-a798-49f7-88a4-f694c2f784da admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:07.930831 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dfa377-a798-49f7-88a4-f694c2f784da admin admin] PATH_INFO: `/v3/users/9ae700f4e75545e2bc0fd5f4d0c6886d` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:07.932066 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfa377-a798-49f7-88a4-f694c2f784da admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ae700f4e75545e2bc0fd5f4d0c6886d)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:07.934071 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dfa377-a798-49f7-88a4-f694c2f784da admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:07.991796 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-c0dfa377-a798-49f7-88a4-f694c2f784da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ae700f4e75545e2bc0fd5f4d0c6886d'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:07.994130 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:08.001461 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-c0dfa377-a798-49f7-88a4-f694c2f784da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ae700f4e75545e2bc0fd5f4d0c6886d'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:08.010184 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-c0dfa377-a798-49f7-88a4-f694c2f784da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ae700f4e75545e2bc0fd5f4d0c6886d'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:08.010877 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-c0dfa377-a798-49f7-88a4-f694c2f784da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ae700f4e75545e2bc0fd5f4d0c6886d'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:08.011486 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-c0dfa377-a798-49f7-88a4-f694c2f784da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ae700f4e75545e2bc0fd5f4d0c6886d'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:08.031442 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1318/2597] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:07 2022] DELETE /identity/v3/users/9ae700f4e75545e2bc0fd5f4d0c6886d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:08.558722 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c594ef3-f5ff-479f-8c0b-5fb42da80a35 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:08.560274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1c594ef3-f5ff-479f-8c0b-5fb42da80a35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:08.575342 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c594ef3-f5ff-479f-8c0b-5fb42da80a35 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:08.634046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018f61e-24ea-41d7-8e39-672e5c0d7e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:08.634881 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018f61e-24ea-41d7-8e39-672e5c0d7e69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:08.635199 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018f61e-24ea-41d7-8e39-672e5c0d7e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:08.635479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018f61e-24ea-41d7-8e39-672e5c0d7e69 admin admin] PATH_INFO: `/v3/projects/61e270874cb54f0ab19153d4171fd2f8` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:08.636707 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018f61e-24ea-41d7-8e39-672e5c0d7e69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61e270874cb54f0ab19153d4171fd2f8)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:08.638637 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018f61e-24ea-41d7-8e39-672e5c0d7e69 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:08.705511 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d018f61e-24ea-41d7-8e39-672e5c0d7e69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61e270874cb54f0ab19153d4171fd2f8', 'request_id': 'req-d018f61e-24ea-41d7-8e39-672e5c0d7e69'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:08.714846 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d018f61e-24ea-41d7-8e39-672e5c0d7e69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61e270874cb54f0ab19153d4171fd2f8', 'request_id': 'req-d018f61e-24ea-41d7-8e39-672e5c0d7e69'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:08.721803 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d018f61e-24ea-41d7-8e39-672e5c0d7e69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61e270874cb54f0ab19153d4171fd2f8', 'request_id': 'req-d018f61e-24ea-41d7-8e39-672e5c0d7e69'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:08.734449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:08.739895 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d018f61e-24ea-41d7-8e39-672e5c0d7e69 admin admin] The token cache is being invalidate because project 61e270874cb54f0ab19153d4171fd2f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:08.740182 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d018f61e-24ea-41d7-8e39-672e5c0d7e69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:08.740789 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-d018f61e-24ea-41d7-8e39-672e5c0d7e69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:08.742325 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1280/2598] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:08 2022] DELETE /identity/v3/projects/61e270874cb54f0ab19153d4171fd2f8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:08.747479 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8479a17-0811-4b24-840c-b15117ae01a8 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:08.748053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8479a17-0811-4b24-840c-b15117ae01a8 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:08.748053 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8479a17-0811-4b24-840c-b15117ae01a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:08.767668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-f8479a17-0811-4b24-840c-b15117ae01a8 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:08.768958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f8479a17-0811-4b24-840c-b15117ae01a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:08.832549 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1319/2599] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:08.841556 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44af28-e160-4594-a369-891a42acf307 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:08.841842 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44af28-e160-4594-a369-891a42acf307 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:08.842116 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44af28-e160-4594-a369-891a42acf307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:08.862386 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-1c44af28-e160-4594-a369-891a42acf307 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:08.863623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1c44af28-e160-4594-a369-891a42acf307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:08.871576 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1281/2600] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:08.875863 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8479a17-0811-4b24-840c-b15117ae01a8 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:08.886753 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8479a17-0811-4b24-840c-b15117ae01a8 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:08.891391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba55970-106f-472e-9c6f-3eebd0f6532e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:08.891994 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba55970-106f-472e-9c6f-3eebd0f6532e admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:08.892271 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba55970-106f-472e-9c6f-3eebd0f6532e admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:08.892542 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba55970-106f-472e-9c6f-3eebd0f6532e admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:08.893179 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba55970-106f-472e-9c6f-3eebd0f6532e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:16:08.894239 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba55970-106f-472e-9c6f-3eebd0f6532e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:08.895890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba55970-106f-472e-9c6f-3eebd0f6532e admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:08.907609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1320/2601] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:08.912101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44af28-e160-4594-a369-891a42acf307 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:08.921115 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44af28-e160-4594-a369-891a42acf307 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:08.925909 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a85c7c-742f-49d3-a59f-803c80ba94ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:08.926507 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a85c7c-742f-49d3-a59f-803c80ba94ef admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:08.926788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a85c7c-742f-49d3-a59f-803c80ba94ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:08.927058 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a85c7c-742f-49d3-a59f-803c80ba94ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:08.927710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a85c7c-742f-49d3-a59f-803c80ba94ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:08.929611 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a85c7c-742f-49d3-a59f-803c80ba94ef admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:08.957766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1282/2602] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:16:08 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:08.962277 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba55970-106f-472e-9c6f-3eebd0f6532e admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:08.972303 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba55970-106f-472e-9c6f-3eebd0f6532e admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:09.030112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5241f49-0d2d-457d-8c8e-3de82e9928e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:09.030726 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5241f49-0d2d-457d-8c8e-3de82e9928e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:09.031007 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5241f49-0d2d-457d-8c8e-3de82e9928e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:09.031325 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5241f49-0d2d-457d-8c8e-3de82e9928e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:09.031964 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5241f49-0d2d-457d-8c8e-3de82e9928e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:09.033882 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5241f49-0d2d-457d-8c8e-3de82e9928e4 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:09.057998 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1321/2603] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:16:08 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:09.059230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a85c7c-742f-49d3-a59f-803c80ba94ef admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:09.070296 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a85c7c-742f-49d3-a59f-803c80ba94ef admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:09.074990 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53512da-7152-41a2-a141-7a37945b71a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:09.075614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53512da-7152-41a2-a141-7a37945b71a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:09.075895 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53512da-7152-41a2-a141-7a37945b71a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:09.076170 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53512da-7152-41a2-a141-7a37945b71a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:09.076810 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53512da-7152-41a2-a141-7a37945b71a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:09.078559 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53512da-7152-41a2-a141-7a37945b71a0 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:09.092047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1283/2604] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:09.094837 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5241f49-0d2d-457d-8c8e-3de82e9928e4 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:09.104905 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5241f49-0d2d-457d-8c8e-3de82e9928e4 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:09.109436 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb286c-455d-47f7-bc38-1d7c3103e134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:09.110025 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb286c-455d-47f7-bc38-1d7c3103e134 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:09.110304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb286c-455d-47f7-bc38-1d7c3103e134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:09.110573 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb286c-455d-47f7-bc38-1d7c3103e134 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:09.111233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb286c-455d-47f7-bc38-1d7c3103e134 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:09.112852 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb286c-455d-47f7-bc38-1d7c3103e134 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:09.123274 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1322/2605] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:09.127116 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53512da-7152-41a2-a141-7a37945b71a0 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:09.137353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53512da-7152-41a2-a141-7a37945b71a0 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:09.141933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062303b-6b8b-4331-b478-ba9da0d30c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:09.142597 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062303b-6b8b-4331-b478-ba9da0d30c90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:09.142874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062303b-6b8b-4331-b478-ba9da0d30c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:09.143178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a062303b-6b8b-4331-b478-ba9da0d30c90 admin admin] PATH_INFO: `/v3/projects/35ea29a380fe45089a9bf240a4e690dd/users/415baefc56854018a2106a5a05380d7f/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:09.158888 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062303b-6b8b-4331-b478-ba9da0d30c90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35ea29a380fe45089a9bf240a4e690dd, user_id=415baefc56854018a2106a5a05380d7f, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:09.161560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a062303b-6b8b-4331-b478-ba9da0d30c90 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:09.173990 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1284/2606] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:09 2022] PUT /identity/v3/projects/35ea29a380fe45089a9bf240a4e690dd/users/415baefc56854018a2106a5a05380d7f/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:09.184565 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938418c7-d4dc-4d1b-b94c-5ed881646860 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:09.184903 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938418c7-d4dc-4d1b-b94c-5ed881646860 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:09.185194 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938418c7-d4dc-4d1b-b94c-5ed881646860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:09.213406 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-938418c7-d4dc-4d1b-b94c-5ed881646860 None None] MFA Rules not processed for user `415baefc56854018a2106a5a05380d7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:09.214691 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-938418c7-d4dc-4d1b-b94c-5ed881646860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:09.235051 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:09.286078 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1323/2607] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:09.290519 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062303b-6b8b-4331-b478-ba9da0d30c90 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:09.361689 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a062303b-6b8b-4331-b478-ba9da0d30c90 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:09.366928 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8663303-887b-408b-9c22-53d8347fe40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:09.367584 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8663303-887b-408b-9c22-53d8347fe40a admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:09.367854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8663303-887b-408b-9c22-53d8347fe40a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:09.368439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8663303-887b-408b-9c22-53d8347fe40a admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:09.368756 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8663303-887b-408b-9c22-53d8347fe40a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:09.370678 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8663303-887b-408b-9c22-53d8347fe40a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:09.406689 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1285/2608] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:16:09 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:09.407777 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938418c7-d4dc-4d1b-b94c-5ed881646860 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:09.418788 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938418c7-d4dc-4d1b-b94c-5ed881646860 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:09.480933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3b86c-07ca-48e3-97a1-a899f77b0e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:09.481573 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3b86c-07ca-48e3-97a1-a899f77b0e1a admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:09.481884 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3b86c-07ca-48e3-97a1-a899f77b0e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:09.482250 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3b86c-07ca-48e3-97a1-a899f77b0e1a admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:09.482905 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3b86c-07ca-48e3-97a1-a899f77b0e1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:09.485042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3b86c-07ca-48e3-97a1-a899f77b0e1a admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:09.511310 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1324/2609] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:16:09 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:09.515245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8663303-887b-408b-9c22-53d8347fe40a admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:09.528201 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8663303-887b-408b-9c22-53d8347fe40a admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:09.539100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d34d4-b23e-4aff-b39c-e275534955fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:09.539731 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d34d4-b23e-4aff-b39c-e275534955fb admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:09.540014 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d34d4-b23e-4aff-b39c-e275534955fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:09.540311 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d34d4-b23e-4aff-b39c-e275534955fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:09.540991 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d34d4-b23e-4aff-b39c-e275534955fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:09.542611 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d34d4-b23e-4aff-b39c-e275534955fb admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:09.568497 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1286/2610] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:09.569500 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3b86c-07ca-48e3-97a1-a899f77b0e1a admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:09.579998 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3b86c-07ca-48e3-97a1-a899f77b0e1a admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:09.585278 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c413fc-bcc3-447c-9602-c45c88d2d1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:09.586028 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c413fc-bcc3-447c-9602-c45c88d2d1bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:09.586378 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c413fc-bcc3-447c-9602-c45c88d2d1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:09.586745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c413fc-bcc3-447c-9602-c45c88d2d1bb admin admin] PATH_INFO: `/v3/projects/b0df503c40f2463084275b75d7dad4d4/users/0fbc9f4dd9ae49668d175107ecceb817/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:09.606959 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c413fc-bcc3-447c-9602-c45c88d2d1bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0df503c40f2463084275b75d7dad4d4, user_id=0fbc9f4dd9ae49668d175107ecceb817, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:09.609654 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c413fc-bcc3-447c-9602-c45c88d2d1bb admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:09.623264 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1325/2611] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:09 2022] PUT /identity/v3/projects/b0df503c40f2463084275b75d7dad4d4/users/0fbc9f4dd9ae49668d175107ecceb817/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:16:09.627170 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d34d4-b23e-4aff-b39c-e275534955fb admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:09.639130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d34d4-b23e-4aff-b39c-e275534955fb admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:09.700513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64738d18-c547-4206-9e1a-e046bbc90118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:09.701128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64738d18-c547-4206-9e1a-e046bbc90118 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:09.701435 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64738d18-c547-4206-9e1a-e046bbc90118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:09.701682 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64738d18-c547-4206-9e1a-e046bbc90118 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:09.702345 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64738d18-c547-4206-9e1a-e046bbc90118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:09.703967 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64738d18-c547-4206-9e1a-e046bbc90118 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:09.714341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1287/2612] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:09.718550 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c413fc-bcc3-447c-9602-c45c88d2d1bb admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:09.731632 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c413fc-bcc3-447c-9602-c45c88d2d1bb admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:09.737008 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ff81e-0e03-4b3d-a492-768054c52f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:09.738356 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ff81e-0e03-4b3d-a492-768054c52f2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:09.738629 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ff81e-0e03-4b3d-a492-768054c52f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:09.738901 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ff81e-0e03-4b3d-a492-768054c52f2d admin admin] PATH_INFO: `/v3/projects/b0df503c40f2463084275b75d7dad4d4/users/0fbc9f4dd9ae49668d175107ecceb817/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:09.740872 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ff81e-0e03-4b3d-a492-768054c52f2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0df503c40f2463084275b75d7dad4d4, user_id=0fbc9f4dd9ae49668d175107ecceb817, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:09.743620 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2ff81e-0e03-4b3d-a492-768054c52f2d admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:09.755425 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1326/2613] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:09 2022] PUT /identity/v3/projects/b0df503c40f2463084275b75d7dad4d4/users/0fbc9f4dd9ae49668d175107ecceb817/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:09.766508 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f31489-98c4-478a-b5e8-51e2ffb43274 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:09.766845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f31489-98c4-478a-b5e8-51e2ffb43274 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:09.767158 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f31489-98c4-478a-b5e8-51e2ffb43274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:09.777812 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:09.810754 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-b1f31489-98c4-478a-b5e8-51e2ffb43274 None None] MFA Rules not processed for user `0fbc9f4dd9ae49668d175107ecceb817`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:09.812040 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b1f31489-98c4-478a-b5e8-51e2ffb43274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:09.887756 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1288/2614] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:09 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:09.899096 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ff81e-0e03-4b3d-a492-768054c52f2d admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:09.901019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6f2ff81e-0e03-4b3d-a492-768054c52f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:09.965621 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ff81e-0e03-4b3d-a492-768054c52f2d admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:09.973176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79739698-8b9d-4571-8892-e40ee448541f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:09.973784 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79739698-8b9d-4571-8892-e40ee448541f service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:09.974063 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79739698-8b9d-4571-8892-e40ee448541f service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:09.974332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79739698-8b9d-4571-8892-e40ee448541f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:09.984311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79739698-8b9d-4571-8892-e40ee448541f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:09.985877 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79739698-8b9d-4571-8892-e40ee448541f service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:09.999423 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1327/2615] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:09 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:16:10.066224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f31489-98c4-478a-b5e8-51e2ffb43274 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:10.132867 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f31489-98c4-478a-b5e8-51e2ffb43274 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:10.137555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a77559-15a0-405d-b9f2-06b5ae707858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:10.138200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a77559-15a0-405d-b9f2-06b5ae707858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:10.138479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a77559-15a0-405d-b9f2-06b5ae707858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:10.138752 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a77559-15a0-405d-b9f2-06b5ae707858 admin admin] PATH_INFO: `/v3/users/415baefc56854018a2106a5a05380d7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:10.139995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a77559-15a0-405d-b9f2-06b5ae707858 admin admin] RBAC: Authorizing `identity:delete_user(user_id=415baefc56854018a2106a5a05380d7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:10.142032 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a77559-15a0-405d-b9f2-06b5ae707858 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:10.188477 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-85a77559-15a0-405d-b9f2-06b5ae707858 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '415baefc56854018a2106a5a05380d7f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:10.189175 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-85a77559-15a0-405d-b9f2-06b5ae707858 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '415baefc56854018a2106a5a05380d7f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:10.201947 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-85a77559-15a0-405d-b9f2-06b5ae707858 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '415baefc56854018a2106a5a05380d7f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:10.208996 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-85a77559-15a0-405d-b9f2-06b5ae707858 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '415baefc56854018a2106a5a05380d7f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:10.209638 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-85a77559-15a0-405d-b9f2-06b5ae707858 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '415baefc56854018a2106a5a05380d7f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:10.222266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1289/2616] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:10 2022] DELETE /identity/v3/users/415baefc56854018a2106a5a05380d7f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:10.226719 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95aad38-5629-45a8-a82c-4cfdb2591c5b None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:10.227002 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95aad38-5629-45a8-a82c-4cfdb2591c5b None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:10.227336 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95aad38-5629-45a8-a82c-4cfdb2591c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:10.246232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-b95aad38-5629-45a8-a82c-4cfdb2591c5b None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:10.247462 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-b95aad38-5629-45a8-a82c-4cfdb2591c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:10.258626 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:10.307102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1328/2617] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:10.317014 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a77559-15a0-405d-b9f2-06b5ae707858 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:10.318466 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-85a77559-15a0-405d-b9f2-06b5ae707858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:10.329201 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:10.384828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a77559-15a0-405d-b9f2-06b5ae707858 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:10.389033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd2f02-bce1-4531-aafd-72afb626d0d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:10.389676 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd2f02-bce1-4531-aafd-72afb626d0d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:10.389950 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd2f02-bce1-4531-aafd-72afb626d0d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:10.390221 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cd2f02-bce1-4531-aafd-72afb626d0d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:10.400491 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd2f02-bce1-4531-aafd-72afb626d0d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:10.401920 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cd2f02-bce1-4531-aafd-72afb626d0d7 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:10.411875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1290/2618] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:16:10.678995 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95aad38-5629-45a8-a82c-4cfdb2591c5b None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:10.680509 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-b95aad38-5629-45a8-a82c-4cfdb2591c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:10.693507 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95aad38-5629-45a8-a82c-4cfdb2591c5b None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:10.697962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60456f55-3313-4b74-ad67-b633989c79a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:10.698604 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60456f55-3313-4b74-ad67-b633989c79a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:10.698885 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60456f55-3313-4b74-ad67-b633989c79a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:10.699188 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60456f55-3313-4b74-ad67-b633989c79a9 admin admin] PATH_INFO: `/v3/projects/35ea29a380fe45089a9bf240a4e690dd` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:10.700310 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60456f55-3313-4b74-ad67-b633989c79a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35ea29a380fe45089a9bf240a4e690dd)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:10.702226 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60456f55-3313-4b74-ad67-b633989c79a9 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:10.750567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-60456f55-3313-4b74-ad67-b633989c79a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35ea29a380fe45089a9bf240a4e690dd', 'request_id': 'req-60456f55-3313-4b74-ad67-b633989c79a9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:10.758627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-60456f55-3313-4b74-ad67-b633989c79a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35ea29a380fe45089a9bf240a4e690dd', 'request_id': 'req-60456f55-3313-4b74-ad67-b633989c79a9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:10.764186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-60456f55-3313-4b74-ad67-b633989c79a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35ea29a380fe45089a9bf240a4e690dd', 'request_id': 'req-60456f55-3313-4b74-ad67-b633989c79a9'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:10.775648 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-60456f55-3313-4b74-ad67-b633989c79a9 admin admin] The token cache is being invalidate because project 35ea29a380fe45089a9bf240a4e690dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:10.775935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-60456f55-3313-4b74-ad67-b633989c79a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:10.776555 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-60456f55-3313-4b74-ad67-b633989c79a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:10.778030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1329/2619] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:10 2022] DELETE /identity/v3/projects/35ea29a380fe45089a9bf240a4e690dd => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:10.781983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd2f02-bce1-4531-aafd-72afb626d0d7 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:10.791605 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cd2f02-bce1-4531-aafd-72afb626d0d7 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:10.852175 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ad41f-d018-4d0a-8dc3-cb365666d17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:10.852802 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ad41f-d018-4d0a-8dc3-cb365666d17b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:10.853081 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ad41f-d018-4d0a-8dc3-cb365666d17b admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:10.853354 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ad41f-d018-4d0a-8dc3-cb365666d17b admin admin] PATH_INFO: `/v3/users/0fbc9f4dd9ae49668d175107ecceb817` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:10.854463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ad41f-d018-4d0a-8dc3-cb365666d17b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fbc9f4dd9ae49668d175107ecceb817)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:10.856411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ad41f-d018-4d0a-8dc3-cb365666d17b admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:10.901661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-e60ad41f-d018-4d0a-8dc3-cb365666d17b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fbc9f4dd9ae49668d175107ecceb817'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:10.902311 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-e60ad41f-d018-4d0a-8dc3-cb365666d17b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fbc9f4dd9ae49668d175107ecceb817'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:10.914282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-e60ad41f-d018-4d0a-8dc3-cb365666d17b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fbc9f4dd9ae49668d175107ecceb817'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:10.921091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-e60ad41f-d018-4d0a-8dc3-cb365666d17b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fbc9f4dd9ae49668d175107ecceb817'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:10.921723 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-e60ad41f-d018-4d0a-8dc3-cb365666d17b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fbc9f4dd9ae49668d175107ecceb817'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:10.935562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1291/2620] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:10 2022] DELETE /identity/v3/users/0fbc9f4dd9ae49668d175107ecceb817 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:11.206614 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60456f55-3313-4b74-ad67-b633989c79a9 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:11.208337 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-60456f55-3313-4b74-ad67-b633989c79a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:11.221230 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60456f55-3313-4b74-ad67-b633989c79a9 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:11.280880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18ab928-1f02-4ab6-812e-fd3968231bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:11.281536 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ab928-1f02-4ab6-812e-fd3968231bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:11.281813 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ab928-1f02-4ab6-812e-fd3968231bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:11.282082 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ab928-1f02-4ab6-812e-fd3968231bb5 admin admin] PATH_INFO: `/v3/projects/b0df503c40f2463084275b75d7dad4d4` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:11.283231 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ab928-1f02-4ab6-812e-fd3968231bb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0df503c40f2463084275b75d7dad4d4)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:11.285151 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18ab928-1f02-4ab6-812e-fd3968231bb5 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:11.328661 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:11.333195 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d18ab928-1f02-4ab6-812e-fd3968231bb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0df503c40f2463084275b75d7dad4d4', 'request_id': 'req-d18ab928-1f02-4ab6-812e-fd3968231bb5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:11.347523 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d18ab928-1f02-4ab6-812e-fd3968231bb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0df503c40f2463084275b75d7dad4d4', 'request_id': 'req-d18ab928-1f02-4ab6-812e-fd3968231bb5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:11.354099 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d18ab928-1f02-4ab6-812e-fd3968231bb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0df503c40f2463084275b75d7dad4d4', 'request_id': 'req-d18ab928-1f02-4ab6-812e-fd3968231bb5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:11.366285 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d18ab928-1f02-4ab6-812e-fd3968231bb5 admin admin] The token cache is being invalidate because project b0df503c40f2463084275b75d7dad4d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:11.366577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d18ab928-1f02-4ab6-812e-fd3968231bb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:11.367279 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d18ab928-1f02-4ab6-812e-fd3968231bb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:11.368770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1330/2621] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:11 2022] DELETE /identity/v3/projects/b0df503c40f2463084275b75d7dad4d4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:11.373880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be81d0-cebd-4bcf-946e-19a6a203209d None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:11.374184 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be81d0-cebd-4bcf-946e-19a6a203209d None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:11.374467 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be81d0-cebd-4bcf-946e-19a6a203209d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:11.395525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-06be81d0-cebd-4bcf-946e-19a6a203209d None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:11.396915 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-06be81d0-cebd-4bcf-946e-19a6a203209d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:11.423175 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:11.460362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1292/2622] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:11.468092 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92de8d-71a2-4d6d-9ddd-5f7fdf2b42b8 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:11.468412 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92de8d-71a2-4d6d-9ddd-5f7fdf2b42b8 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:11.468684 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92de8d-71a2-4d6d-9ddd-5f7fdf2b42b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:11.489281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-bf92de8d-71a2-4d6d-9ddd-5f7fdf2b42b8 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:11.490479 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-bf92de8d-71a2-4d6d-9ddd-5f7fdf2b42b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:11.499588 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1331/2623] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:11.504326 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be81d0-cebd-4bcf-946e-19a6a203209d None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:11.514112 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be81d0-cebd-4bcf-946e-19a6a203209d None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:11.518727 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586880e-91b2-4386-a147-02dfedf646a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:11.519358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586880e-91b2-4386-a147-02dfedf646a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:11.519640 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586880e-91b2-4386-a147-02dfedf646a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:11.519912 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586880e-91b2-4386-a147-02dfedf646a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:11.520554 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586880e-91b2-4386-a147-02dfedf646a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:16:11.520856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586880e-91b2-4386-a147-02dfedf646a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:11.522429 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586880e-91b2-4386-a147-02dfedf646a3 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:11.533702 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1293/2624] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:16:11.538579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92de8d-71a2-4d6d-9ddd-5f7fdf2b42b8 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:11.547695 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92de8d-71a2-4d6d-9ddd-5f7fdf2b42b8 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:11.553377 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27ba82-3952-4f3e-82c6-34f8729e81b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:11.554080 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27ba82-3952-4f3e-82c6-34f8729e81b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:11.554488 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27ba82-3952-4f3e-82c6-34f8729e81b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:11.554793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c27ba82-3952-4f3e-82c6-34f8729e81b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:11.555529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27ba82-3952-4f3e-82c6-34f8729e81b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:11.557857 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c27ba82-3952-4f3e-82c6-34f8729e81b7 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:11.585493 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1332/2625] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:16:11 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:11.589773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586880e-91b2-4386-a147-02dfedf646a3 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:11.600531 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586880e-91b2-4386-a147-02dfedf646a3 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:11.661953 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3015b0-0052-4ac1-aa22-8845fc47fb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:11.662557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3015b0-0052-4ac1-aa22-8845fc47fb17 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:11.662833 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3015b0-0052-4ac1-aa22-8845fc47fb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:11.663132 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3015b0-0052-4ac1-aa22-8845fc47fb17 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:11.663752 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3015b0-0052-4ac1-aa22-8845fc47fb17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:11.665625 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3015b0-0052-4ac1-aa22-8845fc47fb17 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:11.687404 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1294/2626] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:16:11 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:11.691698 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27ba82-3952-4f3e-82c6-34f8729e81b7 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:11.702109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c27ba82-3952-4f3e-82c6-34f8729e81b7 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:11.706929 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43ee1c-430b-49da-b3d4-e042856f05f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:11.707803 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43ee1c-430b-49da-b3d4-e042856f05f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:11.708094 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43ee1c-430b-49da-b3d4-e042856f05f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:11.708356 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43ee1c-430b-49da-b3d4-e042856f05f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:11.709026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43ee1c-430b-49da-b3d4-e042856f05f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:11.710672 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43ee1c-430b-49da-b3d4-e042856f05f9 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:11.720449 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1333/2627] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:11.724709 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3015b0-0052-4ac1-aa22-8845fc47fb17 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:11.734616 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3015b0-0052-4ac1-aa22-8845fc47fb17 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:11.739918 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb23d89-71fc-4bfd-97da-f223998fea48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:11.740507 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb23d89-71fc-4bfd-97da-f223998fea48 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:11.740788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb23d89-71fc-4bfd-97da-f223998fea48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:11.741063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb23d89-71fc-4bfd-97da-f223998fea48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:11.741699 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb23d89-71fc-4bfd-97da-f223998fea48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:11.743318 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb23d89-71fc-4bfd-97da-f223998fea48 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:11.754403 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1295/2628] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:11.758581 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43ee1c-430b-49da-b3d4-e042856f05f9 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:11.768921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43ee1c-430b-49da-b3d4-e042856f05f9 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:11.773830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36250e39-c320-4405-ae32-1a5171c05cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:11.774489 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36250e39-c320-4405-ae32-1a5171c05cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:11.774772 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36250e39-c320-4405-ae32-1a5171c05cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:11.775050 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36250e39-c320-4405-ae32-1a5171c05cc3 admin admin] PATH_INFO: `/v3/projects/83740b213dd14c218cf24c39df06e72e/users/427c80c79b7a41a180f3357d54d9e696/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:11.790308 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36250e39-c320-4405-ae32-1a5171c05cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83740b213dd14c218cf24c39df06e72e, user_id=427c80c79b7a41a180f3357d54d9e696, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:11.793012 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36250e39-c320-4405-ae32-1a5171c05cc3 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:11.806817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1334/2629] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:11 2022] PUT /identity/v3/projects/83740b213dd14c218cf24c39df06e72e/users/427c80c79b7a41a180f3357d54d9e696/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:11.817570 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904ca04-da7d-4cf1-925f-b9fedf202c37 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:11.818104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904ca04-da7d-4cf1-925f-b9fedf202c37 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:11.818104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904ca04-da7d-4cf1-925f-b9fedf202c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:11.847309 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-8904ca04-da7d-4cf1-925f-b9fedf202c37 None None] MFA Rules not processed for user `427c80c79b7a41a180f3357d54d9e696`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:11.848551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-8904ca04-da7d-4cf1-925f-b9fedf202c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:11.913897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1296/2630] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:11.918388 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36250e39-c320-4405-ae32-1a5171c05cc3 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:11.982379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36250e39-c320-4405-ae32-1a5171c05cc3 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:11.986939 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd722ed-e7c8-4a08-9f40-f17f38960c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:11.987604 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd722ed-e7c8-4a08-9f40-f17f38960c68 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:11.987879 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd722ed-e7c8-4a08-9f40-f17f38960c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:11.988146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd722ed-e7c8-4a08-9f40-f17f38960c68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:11.988758 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd722ed-e7c8-4a08-9f40-f17f38960c68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:11.990754 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd722ed-e7c8-4a08-9f40-f17f38960c68 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:12.019087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1335/2631] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:16:11 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:12.023182 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904ca04-da7d-4cf1-925f-b9fedf202c37 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:12.033672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904ca04-da7d-4cf1-925f-b9fedf202c37 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:12.059497 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:12.094049 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6a2de-fe0e-4d6d-b1a1-45e5bd059a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:12.094664 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6a2de-fe0e-4d6d-b1a1-45e5bd059a96 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:12.095007 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6a2de-fe0e-4d6d-b1a1-45e5bd059a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:12.095311 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6a2de-fe0e-4d6d-b1a1-45e5bd059a96 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:12.095938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6a2de-fe0e-4d6d-b1a1-45e5bd059a96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:12.098436 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6a2de-fe0e-4d6d-b1a1-45e5bd059a96 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:12.119118 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1297/2632] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:16:12 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:12.122692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd722ed-e7c8-4a08-9f40-f17f38960c68 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:12.133073 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd722ed-e7c8-4a08-9f40-f17f38960c68 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:12.138081 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170e026-7c57-4984-b4bc-47c7b81bc4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:12.138680 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170e026-7c57-4984-b4bc-47c7b81bc4ab admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:12.138959 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170e026-7c57-4984-b4bc-47c7b81bc4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:12.139269 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170e026-7c57-4984-b4bc-47c7b81bc4ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:12.139909 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170e026-7c57-4984-b4bc-47c7b81bc4ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:12.141496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170e026-7c57-4984-b4bc-47c7b81bc4ab admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:12.153388 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1336/2633] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:12.157390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6a2de-fe0e-4d6d-b1a1-45e5bd059a96 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:12.158759 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170e026-7c57-4984-b4bc-47c7b81bc4ab admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:12.161006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0170e026-7c57-4984-b4bc-47c7b81bc4ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:12.168038 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6a2de-fe0e-4d6d-b1a1-45e5bd059a96 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:12.172661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabc31a-97ec-44d0-8a16-68ec4e004dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:12.173184 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170e026-7c57-4984-b4bc-47c7b81bc4ab admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:12.173304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabc31a-97ec-44d0-8a16-68ec4e004dee admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:12.173574 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabc31a-97ec-44d0-8a16-68ec4e004dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:12.173851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabc31a-97ec-44d0-8a16-68ec4e004dee admin admin] PATH_INFO: `/v3/projects/77224957a06c462a911b00524ad7d3b9/users/3856ed567a35452483ed164003ad7df3/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:12.177658 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdba13-51be-4b82-a1a9-42f57171313c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:12.178268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdba13-51be-4b82-a1a9-42f57171313c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:12.178543 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdba13-51be-4b82-a1a9-42f57171313c admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:12.178813 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdba13-51be-4b82-a1a9-42f57171313c admin admin] PATH_INFO: `/v3/users/690d78909aa540419c08730952431849` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:12.179939 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbdba13-51be-4b82-a1a9-42f57171313c admin admin] RBAC: Authorizing `identity:delete_user(user_id=690d78909aa540419c08730952431849)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:12.181934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbdba13-51be-4b82-a1a9-42f57171313c admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:12.194304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beabc31a-97ec-44d0-8a16-68ec4e004dee admin admin] RBAC: Authorizing `identity:create_grant(project_id=77224957a06c462a911b00524ad7d3b9, user_id=3856ed567a35452483ed164003ad7df3, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:12.196986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beabc31a-97ec-44d0-8a16-68ec4e004dee admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:12.212606 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1298/2634] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:12 2022] PUT /identity/v3/projects/77224957a06c462a911b00524ad7d3b9/users/3856ed567a35452483ed164003ad7df3/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:12.217304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabc31a-97ec-44d0-8a16-68ec4e004dee admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:12.229164 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:12.241841 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-4bbdba13-51be-4b82-a1a9-42f57171313c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '690d78909aa540419c08730952431849'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:12.248734 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-4bbdba13-51be-4b82-a1a9-42f57171313c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '690d78909aa540419c08730952431849'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:12.258005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-4bbdba13-51be-4b82-a1a9-42f57171313c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '690d78909aa540419c08730952431849'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:12.258650 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-4bbdba13-51be-4b82-a1a9-42f57171313c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '690d78909aa540419c08730952431849'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:12.259399 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-4bbdba13-51be-4b82-a1a9-42f57171313c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '690d78909aa540419c08730952431849'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:12.279258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1337/2635] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:12 2022] DELETE /identity/v3/users/690d78909aa540419c08730952431849 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:16:12.283736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b498b-3e89-4199-b9d0-2ed5807bed98 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:12.284003 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b498b-3e89-4199-b9d0-2ed5807bed98 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:12.284378 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7b498b-3e89-4199-b9d0-2ed5807bed98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:12.291697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabc31a-97ec-44d0-8a16-68ec4e004dee admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:12.303113 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-8b7b498b-3e89-4199-b9d0-2ed5807bed98 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:12.304571 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8b7b498b-3e89-4199-b9d0-2ed5807bed98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:12.353070 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812cf836-5d17-4da3-bff1-f23ecbd1cfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:12.353775 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cf836-5d17-4da3-bff1-f23ecbd1cfa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:12.354058 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cf836-5d17-4da3-bff1-f23ecbd1cfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:12.354326 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cf836-5d17-4da3-bff1-f23ecbd1cfa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:12.354968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812cf836-5d17-4da3-bff1-f23ecbd1cfa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:12.356612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812cf836-5d17-4da3-bff1-f23ecbd1cfa8 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:12.367004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1299/2636] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:12.370969 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812cf836-5d17-4da3-bff1-f23ecbd1cfa8 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:12.372621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-812cf836-5d17-4da3-bff1-f23ecbd1cfa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:12.374874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1338/2637] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:16:12.384954 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b498b-3e89-4199-b9d0-2ed5807bed98 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:12.385248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812cf836-5d17-4da3-bff1-f23ecbd1cfa8 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:12.386322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8b7b498b-3e89-4199-b9d0-2ed5807bed98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:12.389929 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9448a05-ae9c-473c-83d9-0a013850d6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:12.390572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9448a05-ae9c-473c-83d9-0a013850d6ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:12.390854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9448a05-ae9c-473c-83d9-0a013850d6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:12.391162 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9448a05-ae9c-473c-83d9-0a013850d6ee admin admin] PATH_INFO: `/v3/projects/77224957a06c462a911b00524ad7d3b9/users/3856ed567a35452483ed164003ad7df3/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:12.393121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9448a05-ae9c-473c-83d9-0a013850d6ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=77224957a06c462a911b00524ad7d3b9, user_id=3856ed567a35452483ed164003ad7df3, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:12.396347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9448a05-ae9c-473c-83d9-0a013850d6ee admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:12.406471 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1300/2638] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:12 2022] PUT /identity/v3/projects/77224957a06c462a911b00524ad7d3b9/users/3856ed567a35452483ed164003ad7df3/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:16:12.420744 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505c287-b2b7-411d-a52c-d5213f8c708c None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:12.420744 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505c287-b2b7-411d-a52c-d5213f8c708c None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:12.420744 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7505c287-b2b7-411d-a52c-d5213f8c708c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:12.447800 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7b498b-3e89-4199-b9d0-2ed5807bed98 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:12.448366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-7505c287-b2b7-411d-a52c-d5213f8c708c None None] MFA Rules not processed for user `3856ed567a35452483ed164003ad7df3`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:12.449710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7505c287-b2b7-411d-a52c-d5213f8c708c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:12.460449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:12.504315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658317bf-f0d4-4b83-9a55-198903d7a1b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:12.504849 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658317bf-f0d4-4b83-9a55-198903d7a1b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:12.505129 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658317bf-f0d4-4b83-9a55-198903d7a1b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:12.505400 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658317bf-f0d4-4b83-9a55-198903d7a1b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:12.528978 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1301/2639] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:12 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:16:12.533830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:12.541890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505c287-b2b7-411d-a52c-d5213f8c708c None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:12.543484 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-7505c287-b2b7-411d-a52c-d5213f8c708c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:12.574586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658317bf-f0d4-4b83-9a55-198903d7a1b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:12.576051 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658317bf-f0d4-4b83-9a55-198903d7a1b6 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:12.586847 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1339/2640] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:16:12.609923 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7505c287-b2b7-411d-a52c-d5213f8c708c None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:12.614122 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ea9a4-c932-4b4e-bd05-7b128a25dbfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:12.614830 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ea9a4-c932-4b4e-bd05-7b128a25dbfd service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:12.615154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ea9a4-c932-4b4e-bd05-7b128a25dbfd service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:12.615425 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ea9a4-c932-4b4e-bd05-7b128a25dbfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:12.629796 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ea9a4-c932-4b4e-bd05-7b128a25dbfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:12.631279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ea9a4-c932-4b4e-bd05-7b128a25dbfd service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:12.641296 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1302/2641] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:12 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:16:12.881855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658317bf-f0d4-4b83-9a55-198903d7a1b6 service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:12.883515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-658317bf-f0d4-4b83-9a55-198903d7a1b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:12.913200 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658317bf-f0d4-4b83-9a55-198903d7a1b6 service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:12.918809 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b681f-b561-434f-ad8b-7ace996c7e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:12.919477 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b681f-b561-434f-ad8b-7ace996c7e8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:12.919758 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b681f-b561-434f-ad8b-7ace996c7e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:12.920032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b681f-b561-434f-ad8b-7ace996c7e8a admin admin] PATH_INFO: `/v3/projects/de94fa3871d740e8add920033f8844df` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:12.921357 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b681f-b561-434f-ad8b-7ace996c7e8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=de94fa3871d740e8add920033f8844df)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:12.923547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b681f-b561-434f-ad8b-7ace996c7e8a admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:12.975137 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9d8b681f-b561-434f-ad8b-7ace996c7e8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de94fa3871d740e8add920033f8844df', 'request_id': 'req-9d8b681f-b561-434f-ad8b-7ace996c7e8a'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:12.983318 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9d8b681f-b561-434f-ad8b-7ace996c7e8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de94fa3871d740e8add920033f8844df', 'request_id': 'req-9d8b681f-b561-434f-ad8b-7ace996c7e8a'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:12.992285 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9d8b681f-b561-434f-ad8b-7ace996c7e8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de94fa3871d740e8add920033f8844df', 'request_id': 'req-9d8b681f-b561-434f-ad8b-7ace996c7e8a'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:12.992716 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ea9a4-c932-4b4e-bd05-7b128a25dbfd service cinder] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:13.004760 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ea9a4-c932-4b4e-bd05-7b128a25dbfd service cinder] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:13.005024 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9d8b681f-b561-434f-ad8b-7ace996c7e8a admin admin] The token cache is being invalidate because project de94fa3871d740e8add920033f8844df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:13.005024 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9d8b681f-b561-434f-ad8b-7ace996c7e8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:13.005330 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-9d8b681f-b561-434f-ad8b-7ace996c7e8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:13.006803 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1340/2642] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:12 2022] DELETE /identity/v3/projects/de94fa3871d740e8add920033f8844df => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:13.011805 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8abe83-ab1e-4bea-8244-dc97a2acd744 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:13.012084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8abe83-ab1e-4bea-8244-dc97a2acd744 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:13.012454 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8abe83-ab1e-4bea-8244-dc97a2acd744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:13.032552 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-df8abe83-ab1e-4bea-8244-dc97a2acd744 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:13.033749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-df8abe83-ab1e-4bea-8244-dc97a2acd744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:13.066318 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0e100-ee44-45d5-ad8c-aba8ec2ea9a5 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3856ed567a35452483ed164003ad7df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77224957a06c462a911b00524ad7d3b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:13.068159 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0e100-ee44-45d5-ad8c-aba8ec2ea9a5 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:13.068446 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0e100-ee44-45d5-ad8c-aba8ec2ea9a5 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:13.068719 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0e100-ee44-45d5-ad8c-aba8ec2ea9a5 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:13.069376 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0e100-ee44-45d5-ad8c-aba8ec2ea9a5 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:16:13.069683 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0e100-ee44-45d5-ad8c-aba8ec2ea9a5 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:13.071310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0e100-ee44-45d5-ad8c-aba8ec2ea9a5 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:13.085074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1303/2643] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:13.089278 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0e100-ee44-45d5-ad8c-aba8ec2ea9a5 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:13.090938 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-71b0e100-ee44-45d5-ad8c-aba8ec2ea9a5 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:13.103634 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0e100-ee44-45d5-ad8c-aba8ec2ea9a5 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:13.113289 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1341/2644] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:16:13.113470 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff000f-12d9-4408-b346-85f689448b99 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3856ed567a35452483ed164003ad7df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77224957a06c462a911b00524ad7d3b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:13.113470 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff000f-12d9-4408-b346-85f689448b99 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:13.113470 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff000f-12d9-4408-b346-85f689448b99 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:13.113772 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bff000f-12d9-4408-b346-85f689448b99 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:13.113772 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff000f-12d9-4408-b346-85f689448b99 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:13.113772 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bff000f-12d9-4408-b346-85f689448b99 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:13.114834 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8fccc-3114-4fce-baa7-4348e712f53a None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:13.115147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8fccc-3114-4fce-baa7-4348e712f53a None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:13.115421 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8fccc-3114-4fce-baa7-4348e712f53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:13.138730 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-bfa8fccc-3114-4fce-baa7-4348e712f53a None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:13.140089 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-bfa8fccc-3114-4fce-baa7-4348e712f53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:13.141253 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1304/2645] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:16:13 2022] POST /identity/v3/projects => generated 354 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:16:13.145495 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff000f-12d9-4408-b346-85f689448b99 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:13.170522 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:13.182895 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:13.208017 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1342/2646] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:16:13.211974 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8fccc-3114-4fce-baa7-4348e712f53a None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:13.216011 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bff000f-12d9-4408-b346-85f689448b99 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:13.220448 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8fccc-3114-4fce-baa7-4348e712f53a None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:13.220925 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c625f69-6b73-4d71-bf73-028dfe4d5e80 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3856ed567a35452483ed164003ad7df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77224957a06c462a911b00524ad7d3b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:13.221538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c625f69-6b73-4d71-bf73-028dfe4d5e80 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:13.222342 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c625f69-6b73-4d71-bf73-028dfe4d5e80 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:13.222620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c625f69-6b73-4d71-bf73-028dfe4d5e80 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:13.223302 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c625f69-6b73-4d71-bf73-028dfe4d5e80 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:16:13.223614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c625f69-6b73-4d71-bf73-028dfe4d5e80 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:13.225214 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c625f69-6b73-4d71-bf73-028dfe4d5e80 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:13.225672 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba60c3-b0b6-4eab-9d07-953a1113f53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:13.226249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba60c3-b0b6-4eab-9d07-953a1113f53e admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:13.226529 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba60c3-b0b6-4eab-9d07-953a1113f53e admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:13.226801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba60c3-b0b6-4eab-9d07-953a1113f53e admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:13.227496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba60c3-b0b6-4eab-9d07-953a1113f53e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:16:13.227796 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba60c3-b0b6-4eab-9d07-953a1113f53e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:13.229366 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba60c3-b0b6-4eab-9d07-953a1113f53e admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:13.237333 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1305/2647] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:16:13.241560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1343/2648] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:16:13.245581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c625f69-6b73-4d71-bf73-028dfe4d5e80 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:13.256738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c625f69-6b73-4d71-bf73-028dfe4d5e80 tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:13.266008 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab2f07-442a-44ec-9a76-5605a44cfd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:13.266560 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab2f07-442a-44ec-9a76-5605a44cfd3a admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:13.266840 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab2f07-442a-44ec-9a76-5605a44cfd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:13.267137 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab2f07-442a-44ec-9a76-5605a44cfd3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:13.267909 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ab2f07-442a-44ec-9a76-5605a44cfd3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:13.270330 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ab2f07-442a-44ec-9a76-5605a44cfd3a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:13.281517 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba60c3-b0b6-4eab-9d07-953a1113f53e admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:13.293670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba60c3-b0b6-4eab-9d07-953a1113f53e admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:13.298339 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f1ff0-b7fe-465d-bf8d-b120cd1ab6cb tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3856ed567a35452483ed164003ad7df3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '77224957a06c462a911b00524ad7d3b9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:13.302271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1306/2649] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:16:13 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:13.302420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f1ff0-b7fe-465d-bf8d-b120cd1ab6cb tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:13.302420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f1ff0-b7fe-465d-bf8d-b120cd1ab6cb tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:13.302420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f1ff0-b7fe-465d-bf8d-b120cd1ab6cb tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] PATH_INFO: `/v3/projects/d10374f73bf9417987e3a4131a0281dc` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:13.302420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f1ff0-b7fe-465d-bf8d-b120cd1ab6cb tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d10374f73bf9417987e3a4131a0281dc)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:13.302784 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f1ff0-b7fe-465d-bf8d-b120cd1ab6cb tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:13.303115 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab2f07-442a-44ec-9a76-5605a44cfd3a admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:13.360618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-584f1ff0-b7fe-465d-bf8d-b120cd1ab6cb tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd10374f73bf9417987e3a4131a0281dc', 'request_id': 'req-584f1ff0-b7fe-465d-bf8d-b120cd1ab6cb'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:13.368598 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-584f1ff0-b7fe-465d-bf8d-b120cd1ab6cb tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd10374f73bf9417987e3a4131a0281dc', 'request_id': 'req-584f1ff0-b7fe-465d-bf8d-b120cd1ab6cb'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:13.371688 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab2f07-442a-44ec-9a76-5605a44cfd3a admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:13.374627 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-584f1ff0-b7fe-465d-bf8d-b120cd1ab6cb tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd10374f73bf9417987e3a4131a0281dc', 'request_id': 'req-584f1ff0-b7fe-465d-bf8d-b120cd1ab6cb'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:13.391406 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-584f1ff0-b7fe-465d-bf8d-b120cd1ab6cb tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] The token cache is being invalidate because project d10374f73bf9417987e3a4131a0281dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:13.391699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-584f1ff0-b7fe-465d-bf8d-b120cd1ab6cb tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:13.392391 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-584f1ff0-b7fe-465d-bf8d-b120cd1ab6cb tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:13.394653 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1344/2650] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:13 2022] DELETE /identity/v3/projects/d10374f73bf9417987e3a4131a0281dc => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:13.434633 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04887857-d0d0-411b-af4e-c8d3921f67c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:13.435281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04887857-d0d0-411b-af4e-c8d3921f67c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:13.435559 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04887857-d0d0-411b-af4e-c8d3921f67c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:13.435829 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04887857-d0d0-411b-af4e-c8d3921f67c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:13.436927 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04887857-d0d0-411b-af4e-c8d3921f67c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:13.438358 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04887857-d0d0-411b-af4e-c8d3921f67c2 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:13.469314 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1307/2651] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:16:13 2022] POST /identity/v3/users => generated 284 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:16:13.470687 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f1ff0-b7fe-465d-bf8d-b120cd1ab6cb tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:13.472823 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-584f1ff0-b7fe-465d-bf8d-b120cd1ab6cb tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:13.486180 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f1ff0-b7fe-465d-bf8d-b120cd1ab6cb tempest-VolumeQuotaClassesTest-1618382422 tempest-VolumeQuotaClassesTest-1618382422-project-admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:13.491093 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bc6b6-8196-4097-b240-b10bafc2f79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:13.491707 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bc6b6-8196-4097-b240-b10bafc2f79d admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:13.491988 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bc6b6-8196-4097-b240-b10bafc2f79d admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:13.492259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bc6b6-8196-4097-b240-b10bafc2f79d admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:13.492892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54bc6b6-8196-4097-b240-b10bafc2f79d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:13.494480 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54bc6b6-8196-4097-b240-b10bafc2f79d admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:13.504977 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1345/2652] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:13.509149 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04887857-d0d0-411b-af4e-c8d3921f67c2 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:13.520281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04887857-d0d0-411b-af4e-c8d3921f67c2 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:13.525135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19a48c5-d4e5-4ace-9b8c-2c28751ee93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:13.525647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a48c5-d4e5-4ace-9b8c-2c28751ee93a admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:13.526133 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a48c5-d4e5-4ace-9b8c-2c28751ee93a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:13.526488 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19a48c5-d4e5-4ace-9b8c-2c28751ee93a admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:13.526820 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a48c5-d4e5-4ace-9b8c-2c28751ee93a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:13.528479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19a48c5-d4e5-4ace-9b8c-2c28751ee93a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:13.540250 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1308/2653] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:13.543702 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bc6b6-8196-4097-b240-b10bafc2f79d admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:13.554537 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bc6b6-8196-4097-b240-b10bafc2f79d admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:13.559302 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90151063-955e-4fc1-8109-63aff006f48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:13.559956 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90151063-955e-4fc1-8109-63aff006f48a admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:13.560233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90151063-955e-4fc1-8109-63aff006f48a admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:13.560505 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90151063-955e-4fc1-8109-63aff006f48a admin admin] PATH_INFO: `/v3/projects/738ce66d0d5c46d6a1c7829a4c941fad/users/3e2f32e5ca814d389077251a57cd41ce/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:13.575899 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90151063-955e-4fc1-8109-63aff006f48a admin admin] RBAC: Authorizing `identity:create_grant(project_id=738ce66d0d5c46d6a1c7829a4c941fad, user_id=3e2f32e5ca814d389077251a57cd41ce, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:13.578636 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90151063-955e-4fc1-8109-63aff006f48a admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:13.595802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1346/2654] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:13 2022] PUT /identity/v3/projects/738ce66d0d5c46d6a1c7829a4c941fad/users/3e2f32e5ca814d389077251a57cd41ce/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:13.606608 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7c14e-33a7-49a5-8a9f-0944979a20a2 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:13.607449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7c14e-33a7-49a5-8a9f-0944979a20a2 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:13.607449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7c14e-33a7-49a5-8a9f-0944979a20a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:13.614805 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90151063-955e-4fc1-8109-63aff006f48a admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:13.616619 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-90151063-955e-4fc1-8109-63aff006f48a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:13.640140 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-c5a7c14e-33a7-49a5-8a9f-0944979a20a2 None None] MFA Rules not processed for user `3e2f32e5ca814d389077251a57cd41ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:13.641363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c5a7c14e-33a7-49a5-8a9f-0944979a20a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:13.650034 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:13.682158 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90151063-955e-4fc1-8109-63aff006f48a admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:13.688067 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f867444a-f101-48ef-9e71-9690d9910efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:13.688688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867444a-f101-48ef-9e71-9690d9910efc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:13.688963 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867444a-f101-48ef-9e71-9690d9910efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:13.689232 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867444a-f101-48ef-9e71-9690d9910efc admin admin] PATH_INFO: `/v3/users/427c80c79b7a41a180f3357d54d9e696` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:13.690864 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867444a-f101-48ef-9e71-9690d9910efc admin admin] RBAC: Authorizing `identity:delete_user(user_id=427c80c79b7a41a180f3357d54d9e696)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:13.692945 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f867444a-f101-48ef-9e71-9690d9910efc admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:13.710101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1309/2655] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:13.717480 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:13.722833 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7c14e-33a7-49a5-8a9f-0944979a20a2 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:13.724337 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c5a7c14e-33a7-49a5-8a9f-0944979a20a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:13.746634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f867444a-f101-48ef-9e71-9690d9910efc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '427c80c79b7a41a180f3357d54d9e696'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:13.754580 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f867444a-f101-48ef-9e71-9690d9910efc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '427c80c79b7a41a180f3357d54d9e696'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:13.770210 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f867444a-f101-48ef-9e71-9690d9910efc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '427c80c79b7a41a180f3357d54d9e696'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:13.770994 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f867444a-f101-48ef-9e71-9690d9910efc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '427c80c79b7a41a180f3357d54d9e696'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:13.772079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f867444a-f101-48ef-9e71-9690d9910efc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '427c80c79b7a41a180f3357d54d9e696'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:13.800167 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1347/2656] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:13 2022] DELETE /identity/v3/users/427c80c79b7a41a180f3357d54d9e696 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:16:13.804025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7c14e-33a7-49a5-8a9f-0944979a20a2 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:13.804618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923162eb-f2ca-4fd8-9e2e-451108a8d771 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:13.804970 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923162eb-f2ca-4fd8-9e2e-451108a8d771 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:13.805269 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923162eb-f2ca-4fd8-9e2e-451108a8d771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:13.824881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-923162eb-f2ca-4fd8-9e2e-451108a8d771 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:13.826100 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-923162eb-f2ca-4fd8-9e2e-451108a8d771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:13.857833 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97952039-fcfa-4004-908b-215293f59f34 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2474d89f10764afbbc5e740dccaae4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:13.858440 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97952039-fcfa-4004-908b-215293f59f34 service glance] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:13.858722 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97952039-fcfa-4004-908b-215293f59f34 service glance] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:13.858986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97952039-fcfa-4004-908b-215293f59f34 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:13.860763 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-97952039-fcfa-4004-908b-215293f59f34 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:13.888222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1348/2657] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:16:13.900810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923162eb-f2ca-4fd8-9e2e-451108a8d771 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:13.902078 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-923162eb-f2ca-4fd8-9e2e-451108a8d771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:13.922521 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97952039-fcfa-4004-908b-215293f59f34 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:13.923981 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97952039-fcfa-4004-908b-215293f59f34 service glance] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:13.933258 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1310/2658] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:16:13.962098 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923162eb-f2ca-4fd8-9e2e-451108a8d771 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:13.965898 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab8a1f-600d-42b4-bd61-c8b00e33ba65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:13.966522 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab8a1f-600d-42b4-bd61-c8b00e33ba65 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:13.966794 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab8a1f-600d-42b4-bd61-c8b00e33ba65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:13.967138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab8a1f-600d-42b4-bd61-c8b00e33ba65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:13.977989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab8a1f-600d-42b4-bd61-c8b00e33ba65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:13.979463 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab8a1f-600d-42b4-bd61-c8b00e33ba65 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:13.989182 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1349/2659] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:16:14.275937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97952039-fcfa-4004-908b-215293f59f34 service glance] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:14.277429 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-97952039-fcfa-4004-908b-215293f59f34 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:14.294387 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97952039-fcfa-4004-908b-215293f59f34 service glance] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:14.300342 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667121c-8d40-4677-83ed-42eaed16f265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:14.301039 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667121c-8d40-4677-83ed-42eaed16f265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:14.301326 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667121c-8d40-4677-83ed-42eaed16f265 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:14.301589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667121c-8d40-4677-83ed-42eaed16f265 admin admin] PATH_INFO: `/v3/projects/83740b213dd14c218cf24c39df06e72e` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:14.302707 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c667121c-8d40-4677-83ed-42eaed16f265 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83740b213dd14c218cf24c39df06e72e)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:14.304669 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c667121c-8d40-4677-83ed-42eaed16f265 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:14.323794 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:14.359641 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c667121c-8d40-4677-83ed-42eaed16f265 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83740b213dd14c218cf24c39df06e72e', 'request_id': 'req-c667121c-8d40-4677-83ed-42eaed16f265'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:14.367549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c667121c-8d40-4677-83ed-42eaed16f265 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83740b213dd14c218cf24c39df06e72e', 'request_id': 'req-c667121c-8d40-4677-83ed-42eaed16f265'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:14.374110 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c667121c-8d40-4677-83ed-42eaed16f265 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83740b213dd14c218cf24c39df06e72e', 'request_id': 'req-c667121c-8d40-4677-83ed-42eaed16f265'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:14.389786 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c667121c-8d40-4677-83ed-42eaed16f265 admin admin] The token cache is being invalidate because project 83740b213dd14c218cf24c39df06e72e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:14.390007 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c667121c-8d40-4677-83ed-42eaed16f265 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:14.390624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c667121c-8d40-4677-83ed-42eaed16f265 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:14.392372 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1311/2660] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:14 2022] DELETE /identity/v3/projects/83740b213dd14c218cf24c39df06e72e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:14.396694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab8a1f-600d-42b4-bd61-c8b00e33ba65 service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:14.407567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab8a1f-600d-42b4-bd61-c8b00e33ba65 service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:14.455462 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:14.473266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3876c19e-42a6-4e17-9e6e-f5b848d52e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:14.473934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876c19e-42a6-4e17-9e6e-f5b848d52e9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:14.474216 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876c19e-42a6-4e17-9e6e-f5b848d52e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:14.474490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3876c19e-42a6-4e17-9e6e-f5b848d52e9c admin admin] PATH_INFO: `/v3/users/3856ed567a35452483ed164003ad7df3` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:14.475635 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876c19e-42a6-4e17-9e6e-f5b848d52e9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3856ed567a35452483ed164003ad7df3)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:14.477605 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3876c19e-42a6-4e17-9e6e-f5b848d52e9c admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:14.522129 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-3876c19e-42a6-4e17-9e6e-f5b848d52e9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3856ed567a35452483ed164003ad7df3'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:14.528859 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-3876c19e-42a6-4e17-9e6e-f5b848d52e9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3856ed567a35452483ed164003ad7df3'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:14.536535 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-3876c19e-42a6-4e17-9e6e-f5b848d52e9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3856ed567a35452483ed164003ad7df3'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:14.537270 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-3876c19e-42a6-4e17-9e6e-f5b848d52e9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3856ed567a35452483ed164003ad7df3'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:14.537864 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-3876c19e-42a6-4e17-9e6e-f5b848d52e9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3856ed567a35452483ed164003ad7df3'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:14.556046 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1350/2661] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:14 2022] DELETE /identity/v3/users/3856ed567a35452483ed164003ad7df3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:14.827664 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667121c-8d40-4677-83ed-42eaed16f265 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:14.829198 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-c667121c-8d40-4677-83ed-42eaed16f265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:14.853307 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667121c-8d40-4677-83ed-42eaed16f265 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:14.909856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b998bd0-958c-4724-bc99-dcbe362829ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:14.910491 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b998bd0-958c-4724-bc99-dcbe362829ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:14.910774 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b998bd0-958c-4724-bc99-dcbe362829ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:14.911037 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b998bd0-958c-4724-bc99-dcbe362829ce admin admin] PATH_INFO: `/v3/projects/77224957a06c462a911b00524ad7d3b9` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:14.912213 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b998bd0-958c-4724-bc99-dcbe362829ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=77224957a06c462a911b00524ad7d3b9)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:14.914110 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b998bd0-958c-4724-bc99-dcbe362829ce admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:14.960325 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4b998bd0-958c-4724-bc99-dcbe362829ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77224957a06c462a911b00524ad7d3b9', 'request_id': 'req-4b998bd0-958c-4724-bc99-dcbe362829ce'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:14.966630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4b998bd0-958c-4724-bc99-dcbe362829ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77224957a06c462a911b00524ad7d3b9', 'request_id': 'req-4b998bd0-958c-4724-bc99-dcbe362829ce'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:14.972857 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4b998bd0-958c-4724-bc99-dcbe362829ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77224957a06c462a911b00524ad7d3b9', 'request_id': 'req-4b998bd0-958c-4724-bc99-dcbe362829ce'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:14.985235 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4b998bd0-958c-4724-bc99-dcbe362829ce admin admin] The token cache is being invalidate because project 77224957a06c462a911b00524ad7d3b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:14.985520 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4b998bd0-958c-4724-bc99-dcbe362829ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:14.986171 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-4b998bd0-958c-4724-bc99-dcbe362829ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:14.987730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1312/2662] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:14 2022] DELETE /identity/v3/projects/77224957a06c462a911b00524ad7d3b9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:14.992811 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951966fd-b0fd-4625-9d61-555d0e1f9b6f None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:14.993330 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951966fd-b0fd-4625-9d61-555d0e1f9b6f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:14.993439 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951966fd-b0fd-4625-9d61-555d0e1f9b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:15.012806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-951966fd-b0fd-4625-9d61-555d0e1f9b6f None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:15.013951 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-951966fd-b0fd-4625-9d61-555d0e1f9b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:15.076505 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1351/2663] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:15.085114 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358e5da-90a4-436f-9efd-f8f93fec79bc None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:15.085396 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358e5da-90a4-436f-9efd-f8f93fec79bc None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:15.085679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358e5da-90a4-436f-9efd-f8f93fec79bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:15.108630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-1358e5da-90a4-436f-9efd-f8f93fec79bc None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:15.109854 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1358e5da-90a4-436f-9efd-f8f93fec79bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:15.117738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1313/2664] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:15.122184 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951966fd-b0fd-4625-9d61-555d0e1f9b6f None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:15.131850 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951966fd-b0fd-4625-9d61-555d0e1f9b6f None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:15.136562 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e6d3f-44f9-4198-ba12-312d50bbaf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:15.137159 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e6d3f-44f9-4198-ba12-312d50bbaf1c admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:15.137437 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e6d3f-44f9-4198-ba12-312d50bbaf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:15.137708 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e6d3f-44f9-4198-ba12-312d50bbaf1c admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:15.138463 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e6d3f-44f9-4198-ba12-312d50bbaf1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:16:15.138796 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e6d3f-44f9-4198-ba12-312d50bbaf1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:15.140424 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065e6d3f-44f9-4198-ba12-312d50bbaf1c admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:15.153046 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1352/2665] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:15.157268 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358e5da-90a4-436f-9efd-f8f93fec79bc None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:15.167513 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358e5da-90a4-436f-9efd-f8f93fec79bc None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:15.172800 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f15d59-4a6b-4efd-a140-a4d7bb0c1156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:15.173400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f15d59-4a6b-4efd-a140-a4d7bb0c1156 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:15.173681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f15d59-4a6b-4efd-a140-a4d7bb0c1156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:15.173974 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f15d59-4a6b-4efd-a140-a4d7bb0c1156 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:15.174634 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f15d59-4a6b-4efd-a140-a4d7bb0c1156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:15.176699 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f15d59-4a6b-4efd-a140-a4d7bb0c1156 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:15.204828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1314/2666] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:16:15 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:16:15.209934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e6d3f-44f9-4198-ba12-312d50bbaf1c admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:15.219770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e6d3f-44f9-4198-ba12-312d50bbaf1c admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:15.266213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:15.285407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a49fe4-01b5-45cb-b6ad-8bb076bda1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:15.286016 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a49fe4-01b5-45cb-b6ad-8bb076bda1f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:15.286291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a49fe4-01b5-45cb-b6ad-8bb076bda1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:15.286556 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a49fe4-01b5-45cb-b6ad-8bb076bda1f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:15.287213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a49fe4-01b5-45cb-b6ad-8bb076bda1f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:15.289330 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a49fe4-01b5-45cb-b6ad-8bb076bda1f6 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:15.310287 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1353/2667] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:16:15 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:15.314572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f15d59-4a6b-4efd-a140-a4d7bb0c1156 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:15.322060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:15.325927 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f15d59-4a6b-4efd-a140-a4d7bb0c1156 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:15.332937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3079e3-fc14-4ab6-814b-19c984b68b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:15.333782 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3079e3-fc14-4ab6-814b-19c984b68b38 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:15.334217 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3079e3-fc14-4ab6-814b-19c984b68b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:15.334658 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3079e3-fc14-4ab6-814b-19c984b68b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:15.335441 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3079e3-fc14-4ab6-814b-19c984b68b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:15.337231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3079e3-fc14-4ab6-814b-19c984b68b38 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:15.348661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1315/2668] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:15.352881 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a49fe4-01b5-45cb-b6ad-8bb076bda1f6 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:15.363261 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a49fe4-01b5-45cb-b6ad-8bb076bda1f6 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:15.367956 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6efb8-9dd3-4568-907b-b82b688ad9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:15.368554 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6efb8-9dd3-4568-907b-b82b688ad9f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:15.368829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6efb8-9dd3-4568-907b-b82b688ad9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:15.369101 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6efb8-9dd3-4568-907b-b82b688ad9f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:15.369733 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6efb8-9dd3-4568-907b-b82b688ad9f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:15.371348 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6efb8-9dd3-4568-907b-b82b688ad9f2 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:15.382728 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1354/2669] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:16:15.386821 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3079e3-fc14-4ab6-814b-19c984b68b38 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:15.397601 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3079e3-fc14-4ab6-814b-19c984b68b38 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:15.402203 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554dae0-e1f6-480c-a37c-329762791add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:15.402862 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554dae0-e1f6-480c-a37c-329762791add admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:15.403178 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554dae0-e1f6-480c-a37c-329762791add admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:15.403445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554dae0-e1f6-480c-a37c-329762791add admin admin] PATH_INFO: `/v3/projects/08bf71cf9e9e43c79d3b6129a60ae873/users/1dd1d0b7a81144e3987e42561b0bb0bd/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:15.419902 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554dae0-e1f6-480c-a37c-329762791add admin admin] RBAC: Authorizing `identity:create_grant(project_id=08bf71cf9e9e43c79d3b6129a60ae873, user_id=1dd1d0b7a81144e3987e42561b0bb0bd, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:15.422587 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554dae0-e1f6-480c-a37c-329762791add admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:15.436045 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1316/2670] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:15 2022] PUT /identity/v3/projects/08bf71cf9e9e43c79d3b6129a60ae873/users/1dd1d0b7a81144e3987e42561b0bb0bd/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:15.447201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b54bd-92e7-4060-8ed1-ef1ab7446d75 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:15.447509 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b54bd-92e7-4060-8ed1-ef1ab7446d75 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:15.447779 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b54bd-92e7-4060-8ed1-ef1ab7446d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:15.476200 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-8a0b54bd-92e7-4060-8ed1-ef1ab7446d75 None None] MFA Rules not processed for user `1dd1d0b7a81144e3987e42561b0bb0bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:15.477378 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8a0b54bd-92e7-4060-8ed1-ef1ab7446d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:15.539208 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1355/2671] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:15.550242 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554dae0-e1f6-480c-a37c-329762791add admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:15.551980 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-d554dae0-e1f6-480c-a37c-329762791add admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:15.622036 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554dae0-e1f6-480c-a37c-329762791add admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:15.625653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40dfd4-9284-4b78-8454-ec27d2d826ec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2474d89f10764afbbc5e740dccaae4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:15.626241 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40dfd4-9284-4b78-8454-ec27d2d826ec service glance] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:15.626520 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40dfd4-9284-4b78-8454-ec27d2d826ec service glance] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:15.626785 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a40dfd4-9284-4b78-8454-ec27d2d826ec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:15.635573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40dfd4-9284-4b78-8454-ec27d2d826ec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:15.637057 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a40dfd4-9284-4b78-8454-ec27d2d826ec service glance] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:15.646207 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1317/2672] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:16:18.124799 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b54bd-92e7-4060-8ed1-ef1ab7446d75 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:18.126190 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8a0b54bd-92e7-4060-8ed1-ef1ab7446d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:18.184165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b54bd-92e7-4060-8ed1-ef1ab7446d75 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:18.187703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456af96a-4689-48c0-9e00-a66e385e0926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:18.188305 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456af96a-4689-48c0-9e00-a66e385e0926 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:18.188579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456af96a-4689-48c0-9e00-a66e385e0926 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:18.188844 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456af96a-4689-48c0-9e00-a66e385e0926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:18.190532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-456af96a-4689-48c0-9e00-a66e385e0926 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:18.251530 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456af96a-4689-48c0-9e00-a66e385e0926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:18.253745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456af96a-4689-48c0-9e00-a66e385e0926 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:18.265641 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1356/2673] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:16:23.219802 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40dfd4-9284-4b78-8454-ec27d2d826ec service glance] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:23.229724 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a40dfd4-9284-4b78-8454-ec27d2d826ec service glance] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:23.233465 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630099a1-0866-4cf3-8195-56fa525c4202 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:23.234063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630099a1-0866-4cf3-8195-56fa525c4202 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:23.234342 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630099a1-0866-4cf3-8195-56fa525c4202 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:23.234669 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630099a1-0866-4cf3-8195-56fa525c4202 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:23.236463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-630099a1-0866-4cf3-8195-56fa525c4202 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:23.307682 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630099a1-0866-4cf3-8195-56fa525c4202 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:23.309431 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630099a1-0866-4cf3-8195-56fa525c4202 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:23.321482 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1318/2674] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:16:25.182124 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456af96a-4689-48c0-9e00-a66e385e0926 service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:25.183626 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-456af96a-4689-48c0-9e00-a66e385e0926 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:25.202824 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456af96a-4689-48c0-9e00-a66e385e0926 service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:25.209905 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc8589-f77f-496e-b6aa-e9804a9427ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:25.210850 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc8589-f77f-496e-b6aa-e9804a9427ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:25.211376 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc8589-f77f-496e-b6aa-e9804a9427ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:25.211853 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbc8589-f77f-496e-b6aa-e9804a9427ec admin admin] PATH_INFO: `/v3/users/23c7dc5188c44ca0a4717c313c1cde1f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:25.213696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc8589-f77f-496e-b6aa-e9804a9427ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=23c7dc5188c44ca0a4717c313c1cde1f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:25.215903 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbc8589-f77f-496e-b6aa-e9804a9427ec admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:25.223442 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:25.282737 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-0cbc8589-f77f-496e-b6aa-e9804a9427ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23c7dc5188c44ca0a4717c313c1cde1f'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:25.290353 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-0cbc8589-f77f-496e-b6aa-e9804a9427ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23c7dc5188c44ca0a4717c313c1cde1f'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:25.300353 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-0cbc8589-f77f-496e-b6aa-e9804a9427ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23c7dc5188c44ca0a4717c313c1cde1f'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:25.301029 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-0cbc8589-f77f-496e-b6aa-e9804a9427ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23c7dc5188c44ca0a4717c313c1cde1f'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:25.302728 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-0cbc8589-f77f-496e-b6aa-e9804a9427ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23c7dc5188c44ca0a4717c313c1cde1f'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:25.326135 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1357/2675] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:25 2022] DELETE /identity/v3/users/23c7dc5188c44ca0a4717c313c1cde1f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:25.330548 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3491b8-06ed-42ea-8e13-5801939654e8 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:25.331680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3491b8-06ed-42ea-8e13-5801939654e8 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:25.331680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3491b8-06ed-42ea-8e13-5801939654e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:25.352477 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-3d3491b8-06ed-42ea-8e13-5801939654e8 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:25.353691 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-3d3491b8-06ed-42ea-8e13-5801939654e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:25.441828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1319/2676] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:25.452153 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc8589-f77f-496e-b6aa-e9804a9427ec admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:25.455482 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0cbc8589-f77f-496e-b6aa-e9804a9427ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:25.545493 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbc8589-f77f-496e-b6aa-e9804a9427ec admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:25.549467 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab54a5-86e7-4dfe-a1db-3b3e2af3da8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:25.550215 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab54a5-86e7-4dfe-a1db-3b3e2af3da8f service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:25.550497 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab54a5-86e7-4dfe-a1db-3b3e2af3da8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:25.550765 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab54a5-86e7-4dfe-a1db-3b3e2af3da8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:25.572338 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab54a5-86e7-4dfe-a1db-3b3e2af3da8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:25.574236 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab54a5-86e7-4dfe-a1db-3b3e2af3da8f service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:25.607015 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1358/2677] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:16:25.984692 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3491b8-06ed-42ea-8e13-5801939654e8 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:25.986279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-3d3491b8-06ed-42ea-8e13-5801939654e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:26.005660 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3491b8-06ed-42ea-8e13-5801939654e8 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:26.010346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64008430-f71a-44c9-8b34-49e5f4b1086a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:26.010982 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64008430-f71a-44c9-8b34-49e5f4b1086a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:26.011295 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64008430-f71a-44c9-8b34-49e5f4b1086a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:26.011564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64008430-f71a-44c9-8b34-49e5f4b1086a admin admin] PATH_INFO: `/v3/projects/39741bba588d488f9ca657815d729437` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:26.012786 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64008430-f71a-44c9-8b34-49e5f4b1086a admin admin] RBAC: Authorizing `identity:delete_project(project_id=39741bba588d488f9ca657815d729437)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:26.015114 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64008430-f71a-44c9-8b34-49e5f4b1086a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:26.017469 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:26.091436 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-64008430-f71a-44c9-8b34-49e5f4b1086a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39741bba588d488f9ca657815d729437', 'request_id': 'req-64008430-f71a-44c9-8b34-49e5f4b1086a'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:26.099046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-64008430-f71a-44c9-8b34-49e5f4b1086a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39741bba588d488f9ca657815d729437', 'request_id': 'req-64008430-f71a-44c9-8b34-49e5f4b1086a'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:26.104207 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-64008430-f71a-44c9-8b34-49e5f4b1086a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39741bba588d488f9ca657815d729437', 'request_id': 'req-64008430-f71a-44c9-8b34-49e5f4b1086a'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:26.131025 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-64008430-f71a-44c9-8b34-49e5f4b1086a admin admin] The token cache is being invalidate because project 39741bba588d488f9ca657815d729437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:26.131457 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-64008430-f71a-44c9-8b34-49e5f4b1086a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:26.132150 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-64008430-f71a-44c9-8b34-49e5f4b1086a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:26.133972 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1320/2678] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:25 2022] DELETE /identity/v3/projects/39741bba588d488f9ca657815d729437 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:29.820050 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab54a5-86e7-4dfe-a1db-3b3e2af3da8f service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:29.822740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-deab54a5-86e7-4dfe-a1db-3b3e2af3da8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:29.842471 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab54a5-86e7-4dfe-a1db-3b3e2af3da8f service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:29.918018 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe58fb-3ba5-4a75-b475-802b9281205d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:29.918646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe58fb-3ba5-4a75-b475-802b9281205d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:29.918925 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe58fb-3ba5-4a75-b475-802b9281205d admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:29.919228 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fe58fb-3ba5-4a75-b475-802b9281205d admin admin] PATH_INFO: `/v3/users/0fd2f1b7788a4095a28f8fb6917c5705` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:29.920499 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fe58fb-3ba5-4a75-b475-802b9281205d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fd2f1b7788a4095a28f8fb6917c5705)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:29.922734 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fe58fb-3ba5-4a75-b475-802b9281205d admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:29.958490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:29.978716 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-70fe58fb-3ba5-4a75-b475-802b9281205d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fd2f1b7788a4095a28f8fb6917c5705'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:29.985816 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-70fe58fb-3ba5-4a75-b475-802b9281205d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fd2f1b7788a4095a28f8fb6917c5705'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:29.993644 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-70fe58fb-3ba5-4a75-b475-802b9281205d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fd2f1b7788a4095a28f8fb6917c5705'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:29.994278 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-70fe58fb-3ba5-4a75-b475-802b9281205d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fd2f1b7788a4095a28f8fb6917c5705'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:29.994907 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-70fe58fb-3ba5-4a75-b475-802b9281205d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fd2f1b7788a4095a28f8fb6917c5705'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:30.012856 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1359/2679] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:29 2022] DELETE /identity/v3/users/0fd2f1b7788a4095a28f8fb6917c5705 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:30.214626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64008430-f71a-44c9-8b34-49e5f4b1086a admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:30.216236 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-64008430-f71a-44c9-8b34-49e5f4b1086a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:30.231104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64008430-f71a-44c9-8b34-49e5f4b1086a admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:30.309046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26806c0d-ace8-4604-8d65-70bac899ae3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:30.309737 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26806c0d-ace8-4604-8d65-70bac899ae3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:30.310019 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26806c0d-ace8-4604-8d65-70bac899ae3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:30.310290 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26806c0d-ace8-4604-8d65-70bac899ae3f admin admin] PATH_INFO: `/v3/projects/d81e4ab123d54ebdbf4165e250967df6` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:30.311439 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26806c0d-ace8-4604-8d65-70bac899ae3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d81e4ab123d54ebdbf4165e250967df6)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:30.313347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26806c0d-ace8-4604-8d65-70bac899ae3f admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:30.369265 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-26806c0d-ace8-4604-8d65-70bac899ae3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd81e4ab123d54ebdbf4165e250967df6', 'request_id': 'req-26806c0d-ace8-4604-8d65-70bac899ae3f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:30.375653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-26806c0d-ace8-4604-8d65-70bac899ae3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd81e4ab123d54ebdbf4165e250967df6', 'request_id': 'req-26806c0d-ace8-4604-8d65-70bac899ae3f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:30.381136 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-26806c0d-ace8-4604-8d65-70bac899ae3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd81e4ab123d54ebdbf4165e250967df6', 'request_id': 'req-26806c0d-ace8-4604-8d65-70bac899ae3f'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:30.393397 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-26806c0d-ace8-4604-8d65-70bac899ae3f admin admin] The token cache is being invalidate because project d81e4ab123d54ebdbf4165e250967df6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:30.393683 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-26806c0d-ace8-4604-8d65-70bac899ae3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:30.394300 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-26806c0d-ace8-4604-8d65-70bac899ae3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:30.395796 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1321/2680] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:30 2022] DELETE /identity/v3/projects/d81e4ab123d54ebdbf4165e250967df6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:30.399789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe58fb-3ba5-4a75-b475-802b9281205d admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:30.401290 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-70fe58fb-3ba5-4a75-b475-802b9281205d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:30.414852 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fe58fb-3ba5-4a75-b475-802b9281205d admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:30.472445 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb194977-96f6-4951-a938-891cdfdfa66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:30.473080 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb194977-96f6-4951-a938-891cdfdfa66a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:30.473360 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb194977-96f6-4951-a938-891cdfdfa66a admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:30.473733 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb194977-96f6-4951-a938-891cdfdfa66a admin admin] PATH_INFO: `/v3/users/54f2eee57d624e9b8448abc4f4ab685d` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:30.474841 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb194977-96f6-4951-a938-891cdfdfa66a admin admin] RBAC: Authorizing `identity:delete_user(user_id=54f2eee57d624e9b8448abc4f4ab685d)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:30.476802 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb194977-96f6-4951-a938-891cdfdfa66a admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:30.525830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-bb194977-96f6-4951-a938-891cdfdfa66a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54f2eee57d624e9b8448abc4f4ab685d'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:30.532458 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-bb194977-96f6-4951-a938-891cdfdfa66a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54f2eee57d624e9b8448abc4f4ab685d'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:30.540196 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-bb194977-96f6-4951-a938-891cdfdfa66a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54f2eee57d624e9b8448abc4f4ab685d'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:30.540800 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-bb194977-96f6-4951-a938-891cdfdfa66a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54f2eee57d624e9b8448abc4f4ab685d'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:30.541398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-bb194977-96f6-4951-a938-891cdfdfa66a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54f2eee57d624e9b8448abc4f4ab685d'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:30.558562 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1360/2681] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:30 2022] DELETE /identity/v3/users/54f2eee57d624e9b8448abc4f4ab685d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:30.733730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26806c0d-ace8-4604-8d65-70bac899ae3f admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:30.735306 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-26806c0d-ace8-4604-8d65-70bac899ae3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:30.748708 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26806c0d-ace8-4604-8d65-70bac899ae3f admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:30.766589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:30.808282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019aef2b-e69e-4332-ae36-eefce9d8fe0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:30.808850 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019aef2b-e69e-4332-ae36-eefce9d8fe0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:30.809131 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019aef2b-e69e-4332-ae36-eefce9d8fe0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:30.809405 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019aef2b-e69e-4332-ae36-eefce9d8fe0b admin admin] PATH_INFO: `/v3/projects/d0ce498c6a6e428fa228a314e5c351c5` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:30.810567 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019aef2b-e69e-4332-ae36-eefce9d8fe0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0ce498c6a6e428fa228a314e5c351c5)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:30.812551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019aef2b-e69e-4332-ae36-eefce9d8fe0b admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:30.863293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-019aef2b-e69e-4332-ae36-eefce9d8fe0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0ce498c6a6e428fa228a314e5c351c5', 'request_id': 'req-019aef2b-e69e-4332-ae36-eefce9d8fe0b'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:30.873374 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-019aef2b-e69e-4332-ae36-eefce9d8fe0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0ce498c6a6e428fa228a314e5c351c5', 'request_id': 'req-019aef2b-e69e-4332-ae36-eefce9d8fe0b'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:30.880897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-019aef2b-e69e-4332-ae36-eefce9d8fe0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0ce498c6a6e428fa228a314e5c351c5', 'request_id': 'req-019aef2b-e69e-4332-ae36-eefce9d8fe0b'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:30.895689 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-019aef2b-e69e-4332-ae36-eefce9d8fe0b admin admin] The token cache is being invalidate because project d0ce498c6a6e428fa228a314e5c351c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:30.896125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-019aef2b-e69e-4332-ae36-eefce9d8fe0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:30.896851 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-019aef2b-e69e-4332-ae36-eefce9d8fe0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:30.899147 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1322/2682] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:30 2022] DELETE /identity/v3/projects/d0ce498c6a6e428fa228a314e5c351c5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:36.841209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb194977-96f6-4951-a938-891cdfdfa66a admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:36.842559 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-bb194977-96f6-4951-a938-891cdfdfa66a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:36.871805 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:36.928948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb194977-96f6-4951-a938-891cdfdfa66a admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:36.976318 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daefc99-a855-4434-8a52-76399a3d09d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:36.976933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daefc99-a855-4434-8a52-76399a3d09d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:36.977212 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daefc99-a855-4434-8a52-76399a3d09d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:36.977482 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daefc99-a855-4434-8a52-76399a3d09d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:36.979246 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-2daefc99-a855-4434-8a52-76399a3d09d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:37.078904 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daefc99-a855-4434-8a52-76399a3d09d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:37.080387 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daefc99-a855-4434-8a52-76399a3d09d4 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:37.100024 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1361/2683] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:16:39.670989 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019aef2b-e69e-4332-ae36-eefce9d8fe0b admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:39.672567 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-019aef2b-e69e-4332-ae36-eefce9d8fe0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:39.777289 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019aef2b-e69e-4332-ae36-eefce9d8fe0b admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:39.785277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed70e53-3777-4cba-8b58-1f933f10e147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:39.788166 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed70e53-3777-4cba-8b58-1f933f10e147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:39.788571 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed70e53-3777-4cba-8b58-1f933f10e147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:39.788871 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed70e53-3777-4cba-8b58-1f933f10e147 admin admin] PATH_INFO: `/v3/users/1dd1d0b7a81144e3987e42561b0bb0bd` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:39.791629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed70e53-3777-4cba-8b58-1f933f10e147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dd1d0b7a81144e3987e42561b0bb0bd)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:39.793791 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed70e53-3777-4cba-8b58-1f933f10e147 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:39.852165 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9ed70e53-3777-4cba-8b58-1f933f10e147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dd1d0b7a81144e3987e42561b0bb0bd'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:39.853094 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9ed70e53-3777-4cba-8b58-1f933f10e147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dd1d0b7a81144e3987e42561b0bb0bd'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:39.859268 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:39.870280 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9ed70e53-3777-4cba-8b58-1f933f10e147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dd1d0b7a81144e3987e42561b0bb0bd'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:39.879280 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9ed70e53-3777-4cba-8b58-1f933f10e147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dd1d0b7a81144e3987e42561b0bb0bd'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:39.880019 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9ed70e53-3777-4cba-8b58-1f933f10e147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dd1d0b7a81144e3987e42561b0bb0bd'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:39.902045 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1323/2684] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:39 2022] DELETE /identity/v3/users/1dd1d0b7a81144e3987e42561b0bb0bd => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:39.906513 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac224b5-74e6-4077-b391-a68975dff71c None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:39.907006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac224b5-74e6-4077-b391-a68975dff71c None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:39.907123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac224b5-74e6-4077-b391-a68975dff71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:39.933668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:39.936911 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-5ac224b5-74e6-4077-b391-a68975dff71c None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:39.938677 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5ac224b5-74e6-4077-b391-a68975dff71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:39.998561 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1362/2685] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:16:40.008866 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed70e53-3777-4cba-8b58-1f933f10e147 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:40.010419 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-9ed70e53-3777-4cba-8b58-1f933f10e147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:40.070266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed70e53-3777-4cba-8b58-1f933f10e147 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:40.073891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4782bb-a87c-4267-88ef-f3ce83a95891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:40.074608 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4782bb-a87c-4267-88ef-f3ce83a95891 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:40.074914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4782bb-a87c-4267-88ef-f3ce83a95891 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:40.075214 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4782bb-a87c-4267-88ef-f3ce83a95891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:40.087215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4782bb-a87c-4267-88ef-f3ce83a95891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:40.088672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4782bb-a87c-4267-88ef-f3ce83a95891 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:40.099033 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1324/2686] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:16:40.297862 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac224b5-74e6-4077-b391-a68975dff71c None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:40.299353 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5ac224b5-74e6-4077-b391-a68975dff71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:40.313754 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac224b5-74e6-4077-b391-a68975dff71c None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:40.319734 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aef244-376d-49a4-af16-d0c616ac2ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:40.320397 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aef244-376d-49a4-af16-d0c616ac2ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:40.320672 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aef244-376d-49a4-af16-d0c616ac2ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:40.320939 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aef244-376d-49a4-af16-d0c616ac2ecc admin admin] PATH_INFO: `/v3/users/10fcb673c54e484a88a0dabe7b8edb15` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:40.322242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aef244-376d-49a4-af16-d0c616ac2ecc admin admin] RBAC: Authorizing `identity:delete_user(user_id=10fcb673c54e484a88a0dabe7b8edb15)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:40.324249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aef244-376d-49a4-af16-d0c616ac2ecc admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:40.388456 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e9aef244-376d-49a4-af16-d0c616ac2ecc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10fcb673c54e484a88a0dabe7b8edb15'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:40.394436 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4782bb-a87c-4267-88ef-f3ce83a95891 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:40.395930 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-5e4782bb-a87c-4267-88ef-f3ce83a95891 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:40.402829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e9aef244-376d-49a4-af16-d0c616ac2ecc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10fcb673c54e484a88a0dabe7b8edb15'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:40.412969 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4782bb-a87c-4267-88ef-f3ce83a95891 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:40.413396 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e9aef244-376d-49a4-af16-d0c616ac2ecc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10fcb673c54e484a88a0dabe7b8edb15'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:40.414715 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e9aef244-376d-49a4-af16-d0c616ac2ecc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10fcb673c54e484a88a0dabe7b8edb15'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:40.415416 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e9aef244-376d-49a4-af16-d0c616ac2ecc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10fcb673c54e484a88a0dabe7b8edb15'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:40.418389 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9711c-39f4-4b9d-b9c7-78272888b0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:40.419601 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9711c-39f4-4b9d-b9c7-78272888b0ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:40.419953 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9711c-39f4-4b9d-b9c7-78272888b0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:40.420294 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9711c-39f4-4b9d-b9c7-78272888b0ab admin admin] PATH_INFO: `/v3/projects/08bf71cf9e9e43c79d3b6129a60ae873` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:40.422118 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9711c-39f4-4b9d-b9c7-78272888b0ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=08bf71cf9e9e43c79d3b6129a60ae873)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:40.424191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9711c-39f4-4b9d-b9c7-78272888b0ab admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:40.447178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1363/2687] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:40 2022] DELETE /identity/v3/users/10fcb673c54e484a88a0dabe7b8edb15 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:40.482631 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b3b9711c-39f4-4b9d-b9c7-78272888b0ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08bf71cf9e9e43c79d3b6129a60ae873', 'request_id': 'req-b3b9711c-39f4-4b9d-b9c7-78272888b0ab'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:40.496068 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b3b9711c-39f4-4b9d-b9c7-78272888b0ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08bf71cf9e9e43c79d3b6129a60ae873', 'request_id': 'req-b3b9711c-39f4-4b9d-b9c7-78272888b0ab'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:40.510499 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b3b9711c-39f4-4b9d-b9c7-78272888b0ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08bf71cf9e9e43c79d3b6129a60ae873', 'request_id': 'req-b3b9711c-39f4-4b9d-b9c7-78272888b0ab'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:40.526436 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b3b9711c-39f4-4b9d-b9c7-78272888b0ab admin admin] The token cache is being invalidate because project 08bf71cf9e9e43c79d3b6129a60ae873 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:40.527284 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b3b9711c-39f4-4b9d-b9c7-78272888b0ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:40.527922 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-b3b9711c-39f4-4b9d-b9c7-78272888b0ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:40.529433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1325/2688] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:40 2022] DELETE /identity/v3/projects/08bf71cf9e9e43c79d3b6129a60ae873 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:16:40.535018 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e1c15-8a9d-42c4-ba5c-0f61de82f1a1 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:40.535350 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e1c15-8a9d-42c4-ba5c-0f61de82f1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:40.535644 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e1c15-8a9d-42c4-ba5c-0f61de82f1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:40.586004 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-b19e1c15-8a9d-42c4-ba5c-0f61de82f1a1 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:40.587243 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-b19e1c15-8a9d-42c4-ba5c-0f61de82f1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:40.630965 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9711c-39f4-4b9d-b9c7-78272888b0ab admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:40.637876 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b3b9711c-39f4-4b9d-b9c7-78272888b0ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:40.647332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:40.664381 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1364/2689] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:40.672010 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86450da0-cede-43cc-9b70-9e4acfe9317f None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:40.672372 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86450da0-cede-43cc-9b70-9e4acfe9317f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:40.672565 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86450da0-cede-43cc-9b70-9e4acfe9317f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:40.694750 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-86450da0-cede-43cc-9b70-9e4acfe9317f None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:40.696024 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-86450da0-cede-43cc-9b70-9e4acfe9317f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:40.702536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:40.704281 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1365/2690] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:40.708610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86450da0-cede-43cc-9b70-9e4acfe9317f None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:40.709594 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9711c-39f4-4b9d-b9c7-78272888b0ab admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:40.715748 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfed5d1-2fbe-442d-a171-e868eda8f085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:40.716501 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfed5d1-2fbe-442d-a171-e868eda8f085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:40.716862 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfed5d1-2fbe-442d-a171-e868eda8f085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:40.717299 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfed5d1-2fbe-442d-a171-e868eda8f085 admin admin] PATH_INFO: `/v3/projects/e03f09713c2e45d9a261416a5dbd0883` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:40.718182 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86450da0-cede-43cc-9b70-9e4acfe9317f None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:40.718453 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfed5d1-2fbe-442d-a171-e868eda8f085 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e03f09713c2e45d9a261416a5dbd0883)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:40.720458 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfed5d1-2fbe-442d-a171-e868eda8f085 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:40.723343 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c9ef0-6602-4986-96f8-9d9f7efa1044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:40.723922 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c9ef0-6602-4986-96f8-9d9f7efa1044 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:40.724195 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c9ef0-6602-4986-96f8-9d9f7efa1044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:40.724458 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c9ef0-6602-4986-96f8-9d9f7efa1044 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:40.725083 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c9ef0-6602-4986-96f8-9d9f7efa1044 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:16:40.725378 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c9ef0-6602-4986-96f8-9d9f7efa1044 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:40.726952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c9ef0-6602-4986-96f8-9d9f7efa1044 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:40.739332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1366/2691] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:16:40.743584 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c9ef0-6602-4986-96f8-9d9f7efa1044 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:40.753355 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c9ef0-6602-4986-96f8-9d9f7efa1044 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:40.758851 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac471a8f-7f7e-4172-8271-3fd3c1eb87d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:40.759420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac471a8f-7f7e-4172-8271-3fd3c1eb87d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:40.759701 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac471a8f-7f7e-4172-8271-3fd3c1eb87d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:40.759993 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac471a8f-7f7e-4172-8271-3fd3c1eb87d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:40.760665 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac471a8f-7f7e-4172-8271-3fd3c1eb87d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:40.762543 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac471a8f-7f7e-4172-8271-3fd3c1eb87d9 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:40.769868 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-dcfed5d1-2fbe-442d-a171-e868eda8f085 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e03f09713c2e45d9a261416a5dbd0883', 'request_id': 'req-dcfed5d1-2fbe-442d-a171-e868eda8f085'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:40.776035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-dcfed5d1-2fbe-442d-a171-e868eda8f085 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e03f09713c2e45d9a261416a5dbd0883', 'request_id': 'req-dcfed5d1-2fbe-442d-a171-e868eda8f085'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:40.782214 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-dcfed5d1-2fbe-442d-a171-e868eda8f085 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e03f09713c2e45d9a261416a5dbd0883', 'request_id': 'req-dcfed5d1-2fbe-442d-a171-e868eda8f085'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:40.795150 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1367/2692] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:16:40 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:16:40.797308 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-dcfed5d1-2fbe-442d-a171-e868eda8f085 admin admin] The token cache is being invalidate because project e03f09713c2e45d9a261416a5dbd0883 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:40.797609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-dcfed5d1-2fbe-442d-a171-e868eda8f085 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:40.797888 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac471a8f-7f7e-4172-8271-3fd3c1eb87d9 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:40.798268 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-dcfed5d1-2fbe-442d-a171-e868eda8f085 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:40.800421 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1326/2693] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:40 2022] DELETE /identity/v3/projects/e03f09713c2e45d9a261416a5dbd0883 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:16:40.804919 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2e0bf-3740-444d-8ad0-1e403af3221b None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:40.805346 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2e0bf-3740-444d-8ad0-1e403af3221b None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:40.805538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2e0bf-3740-444d-8ad0-1e403af3221b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:40.826015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-63a2e0bf-3740-444d-8ad0-1e403af3221b None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:40.827246 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-63a2e0bf-3740-444d-8ad0-1e403af3221b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:40.870576 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac471a8f-7f7e-4172-8271-3fd3c1eb87d9 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:40.875579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19391f8-913b-4ba4-8aaa-891c0a30d8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:40.876192 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19391f8-913b-4ba4-8aaa-891c0a30d8b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:40.876479 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19391f8-913b-4ba4-8aaa-891c0a30d8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:40.876752 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19391f8-913b-4ba4-8aaa-891c0a30d8b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:40.877382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19391f8-913b-4ba4-8aaa-891c0a30d8b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:40.879369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19391f8-913b-4ba4-8aaa-891c0a30d8b1 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:40.892553 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1327/2694] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:16:40.901035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e175c47-3b6d-4905-b471-47dcaca1b322 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:40.901626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e175c47-3b6d-4905-b471-47dcaca1b322 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:40.902028 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e175c47-3b6d-4905-b471-47dcaca1b322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:40.905534 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1368/2695] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:16:40 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:16:40.906527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19391f8-913b-4ba4-8aaa-891c0a30d8b1 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:40.908420 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a19391f8-913b-4ba4-8aaa-891c0a30d8b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:40.921804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19391f8-913b-4ba4-8aaa-891c0a30d8b1 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:40.924418 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-2e175c47-3b6d-4905-b471-47dcaca1b322 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:40.925696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-2e175c47-3b6d-4905-b471-47dcaca1b322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:40.927736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028b1dd-5a2a-45c2-b289-4347816703c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:40.928346 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028b1dd-5a2a-45c2-b289-4347816703c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:40.928680 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028b1dd-5a2a-45c2-b289-4347816703c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:40.928884 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028b1dd-5a2a-45c2-b289-4347816703c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:40.929590 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028b1dd-5a2a-45c2-b289-4347816703c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:40.931260 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028b1dd-5a2a-45c2-b289-4347816703c9 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:40.935616 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1328/2696] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:16:40.940105 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e175c47-3b6d-4905-b471-47dcaca1b322 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:40.942980 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1369/2697] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:40.947418 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028b1dd-5a2a-45c2-b289-4347816703c9 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:40.953580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e175c47-3b6d-4905-b471-47dcaca1b322 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:40.958814 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028b1dd-5a2a-45c2-b289-4347816703c9 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:40.959337 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8198a1e-6e96-4537-bba5-cd27eb48b44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:40.960063 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8198a1e-6e96-4537-bba5-cd27eb48b44a admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:40.960385 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8198a1e-6e96-4537-bba5-cd27eb48b44a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:40.960660 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8198a1e-6e96-4537-bba5-cd27eb48b44a admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:40.961315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8198a1e-6e96-4537-bba5-cd27eb48b44a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:16:40.961615 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8198a1e-6e96-4537-bba5-cd27eb48b44a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:40.963288 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8198a1e-6e96-4537-bba5-cd27eb48b44a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:40.964890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de8582-7adc-43f7-8e9b-370b77b106ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:40.965670 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de8582-7adc-43f7-8e9b-370b77b106ea admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:40.965958 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de8582-7adc-43f7-8e9b-370b77b106ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:40.966222 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28de8582-7adc-43f7-8e9b-370b77b106ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:40.966856 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de8582-7adc-43f7-8e9b-370b77b106ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:40.969049 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28de8582-7adc-43f7-8e9b-370b77b106ea admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:40.979097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1329/2698] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:40.982713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1370/2699] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:40.982905 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8198a1e-6e96-4537-bba5-cd27eb48b44a admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:40.984125 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de8582-7adc-43f7-8e9b-370b77b106ea admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:40.991449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8198a1e-6e96-4537-bba5-cd27eb48b44a admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:40.993767 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28de8582-7adc-43f7-8e9b-370b77b106ea admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:40.996149 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b534447-bd97-4d21-b71b-5e507c78da74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:40.996741 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b534447-bd97-4d21-b71b-5e507c78da74 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:40.997023 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b534447-bd97-4d21-b71b-5e507c78da74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:40.997290 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b534447-bd97-4d21-b71b-5e507c78da74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:40.997917 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b534447-bd97-4d21-b71b-5e507c78da74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:40.998441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94995a0-e5c1-42e3-9f49-f54b6eda04ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:40.999095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94995a0-e5c1-42e3-9f49-f54b6eda04ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:40.999380 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94995a0-e5c1-42e3-9f49-f54b6eda04ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:40.999653 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94995a0-e5c1-42e3-9f49-f54b6eda04ff admin admin] PATH_INFO: `/v3/projects/5e1246566ce046ef933251215b242a45/users/a39a21bdfd944434a8b60fa90f9553e4/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:40.999833 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b534447-bd97-4d21-b71b-5e507c78da74 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:41.014661 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94995a0-e5c1-42e3-9f49-f54b6eda04ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e1246566ce046ef933251215b242a45, user_id=a39a21bdfd944434a8b60fa90f9553e4, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:41.017300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94995a0-e5c1-42e3-9f49-f54b6eda04ff admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:41.028111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1330/2700] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:16:40 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:41.030366 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1371/2701] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:40 2022] PUT /identity/v3/projects/5e1246566ce046ef933251215b242a45/users/a39a21bdfd944434a8b60fa90f9553e4/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:16:41.036004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b534447-bd97-4d21-b71b-5e507c78da74 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:41.040282 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a191e-af0f-4c23-b2d9-1f309b7c79dc None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:41.040569 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a191e-af0f-4c23-b2d9-1f309b7c79dc None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:41.040844 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a191e-af0f-4c23-b2d9-1f309b7c79dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:41.068452 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-a71a191e-af0f-4c23-b2d9-1f309b7c79dc None None] MFA Rules not processed for user `a39a21bdfd944434a8b60fa90f9553e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:41.069713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a71a191e-af0f-4c23-b2d9-1f309b7c79dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:41.082224 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:41.100632 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b534447-bd97-4d21-b71b-5e507c78da74 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:41.105511 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28417a02-77b0-4f0b-9f7b-3c2c5c0b657f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:41.106115 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28417a02-77b0-4f0b-9f7b-3c2c5c0b657f admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:41.106393 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28417a02-77b0-4f0b-9f7b-3c2c5c0b657f admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:41.106804 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28417a02-77b0-4f0b-9f7b-3c2c5c0b657f admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:41.107666 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28417a02-77b0-4f0b-9f7b-3c2c5c0b657f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:41.109581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28417a02-77b0-4f0b-9f7b-3c2c5c0b657f admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:41.114622 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:41.136357 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1331/2702] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:16:41 2022] POST /identity/v3/users => generated 302 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:41.137008 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1372/2703] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:16:41.139477 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28417a02-77b0-4f0b-9f7b-3c2c5c0b657f admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:41.148859 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a191e-af0f-4c23-b2d9-1f309b7c79dc None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:41.150014 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28417a02-77b0-4f0b-9f7b-3c2c5c0b657f admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:41.150220 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a71a191e-af0f-4c23-b2d9-1f309b7c79dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:41.154840 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aeaedf-0c4c-44e9-83de-ef095a601d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:41.155554 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aeaedf-0c4c-44e9-83de-ef095a601d53 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:41.155837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aeaedf-0c4c-44e9-83de-ef095a601d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:41.156114 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aeaedf-0c4c-44e9-83de-ef095a601d53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:41.156758 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aeaedf-0c4c-44e9-83de-ef095a601d53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:41.158338 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aeaedf-0c4c-44e9-83de-ef095a601d53 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:41.169675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1332/2704] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:41.173574 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aeaedf-0c4c-44e9-83de-ef095a601d53 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:41.182953 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aeaedf-0c4c-44e9-83de-ef095a601d53 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:41.187650 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2a918-3b02-434f-bf5d-b3f542f6be76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:41.188227 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2a918-3b02-434f-bf5d-b3f542f6be76 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:41.188509 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2a918-3b02-434f-bf5d-b3f542f6be76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:41.188782 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2a918-3b02-434f-bf5d-b3f542f6be76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:41.189417 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2a918-3b02-434f-bf5d-b3f542f6be76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:41.191009 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2a918-3b02-434f-bf5d-b3f542f6be76 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:41.201356 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1333/2705] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:16:41.205269 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2a918-3b02-434f-bf5d-b3f542f6be76 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:41.211936 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a191e-af0f-4c23-b2d9-1f309b7c79dc None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:41.215515 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2a918-3b02-434f-bf5d-b3f542f6be76 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:41.215703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37034e6-0383-4514-9725-9a0cc188eec4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2474d89f10764afbbc5e740dccaae4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:41.216312 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37034e6-0383-4514-9725-9a0cc188eec4 service glance] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:41.216589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37034e6-0383-4514-9725-9a0cc188eec4 service glance] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:41.216865 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37034e6-0383-4514-9725-9a0cc188eec4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:41.220260 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15179ef9-f926-4165-9566-58f2c0a3b053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:41.220896 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15179ef9-f926-4165-9566-58f2c0a3b053 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:41.221179 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15179ef9-f926-4165-9566-58f2c0a3b053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:41.221479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15179ef9-f926-4165-9566-58f2c0a3b053 admin admin] PATH_INFO: `/v3/projects/33d80eeac5254e048acbd6bbf362c52b/users/5eb01b576f41426ab8c09dcdbbce4737/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:41.226832 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37034e6-0383-4514-9725-9a0cc188eec4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:41.228355 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37034e6-0383-4514-9725-9a0cc188eec4 service glance] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:41.236837 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15179ef9-f926-4165-9566-58f2c0a3b053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33d80eeac5254e048acbd6bbf362c52b, user_id=5eb01b576f41426ab8c09dcdbbce4737, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:41.237681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1373/2706] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:16:41.239539 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15179ef9-f926-4165-9566-58f2c0a3b053 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:41.252967 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1334/2707] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:41 2022] PUT /identity/v3/projects/33d80eeac5254e048acbd6bbf362c52b/users/5eb01b576f41426ab8c09dcdbbce4737/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:16:41.263335 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4ef8e-143b-47b4-a8ca-edef7e7b4869 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:41.263620 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4ef8e-143b-47b4-a8ca-edef7e7b4869 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:41.263888 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e4ef8e-143b-47b4-a8ca-edef7e7b4869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:41.417475 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-f8e4ef8e-143b-47b4-a8ca-edef7e7b4869 None None] MFA Rules not processed for user `5eb01b576f41426ab8c09dcdbbce4737`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:41.418683 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f8e4ef8e-143b-47b4-a8ca-edef7e7b4869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:41.484669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1374/2708] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:41 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:41.489217 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15179ef9-f926-4165-9566-58f2c0a3b053 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:41.554648 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15179ef9-f926-4165-9566-58f2c0a3b053 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:41.559386 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd9d2c-c8f5-43f8-bc40-85e68e07b8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:41.560125 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd9d2c-c8f5-43f8-bc40-85e68e07b8cd admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:41.560537 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd9d2c-c8f5-43f8-bc40-85e68e07b8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:41.560893 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd9d2c-c8f5-43f8-bc40-85e68e07b8cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:41.561596 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd9d2c-c8f5-43f8-bc40-85e68e07b8cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:41.563591 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd9d2c-c8f5-43f8-bc40-85e68e07b8cd admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:41.594668 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1335/2709] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:16:41 2022] POST /identity/v3/projects => generated 405 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:41.595822 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4ef8e-143b-47b4-a8ca-edef7e7b4869 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:41.606643 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e4ef8e-143b-47b4-a8ca-edef7e7b4869 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:41.680853 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d651b-ee79-409d-a783-e7ed54e759f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:41.681469 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d651b-ee79-409d-a783-e7ed54e759f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:41.681753 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d651b-ee79-409d-a783-e7ed54e759f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:41.682032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7d651b-ee79-409d-a783-e7ed54e759f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:41.682673 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d651b-ee79-409d-a783-e7ed54e759f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:41.684603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7d651b-ee79-409d-a783-e7ed54e759f1 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:41.710327 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1375/2710] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:16:41 2022] POST /identity/v3/users => generated 309 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:16:41.711537 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd9d2c-c8f5-43f8-bc40-85e68e07b8cd admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:41.722083 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd9d2c-c8f5-43f8-bc40-85e68e07b8cd admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:41.726963 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70ab3f-582e-4f7b-b452-f0890cb3c6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:41.727599 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70ab3f-582e-4f7b-b452-f0890cb3c6b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:41.727882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70ab3f-582e-4f7b-b452-f0890cb3c6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:41.728163 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70ab3f-582e-4f7b-b452-f0890cb3c6b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:41.728840 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f70ab3f-582e-4f7b-b452-f0890cb3c6b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:41.730427 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f70ab3f-582e-4f7b-b452-f0890cb3c6b8 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:41.741572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1336/2711] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:41.745823 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d651b-ee79-409d-a783-e7ed54e759f1 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:41.756423 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7d651b-ee79-409d-a783-e7ed54e759f1 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:41.760952 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61961d4e-2b09-4e0e-850d-0ca19cecbdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:41.761593 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61961d4e-2b09-4e0e-850d-0ca19cecbdb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:41.761866 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61961d4e-2b09-4e0e-850d-0ca19cecbdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:41.762132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61961d4e-2b09-4e0e-850d-0ca19cecbdb5 admin admin] PATH_INFO: `/v3/projects/cbcb778f640c4d6b902a3782190e4fc6/users/7b8a0963c4504b198dd7e9da61cb387c/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:41.765142 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:41.778453 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61961d4e-2b09-4e0e-850d-0ca19cecbdb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbcb778f640c4d6b902a3782190e4fc6, user_id=7b8a0963c4504b198dd7e9da61cb387c, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:41.781037 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61961d4e-2b09-4e0e-850d-0ca19cecbdb5 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:41.793169 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1376/2712] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:41 2022] PUT /identity/v3/projects/cbcb778f640c4d6b902a3782190e4fc6/users/7b8a0963c4504b198dd7e9da61cb387c/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:41.797143 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70ab3f-582e-4f7b-b452-f0890cb3c6b8 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:41.807966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70ab3f-582e-4f7b-b452-f0890cb3c6b8 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:41.866117 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac31e8-8f5b-4264-9a88-a84a62b22081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:41.866753 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac31e8-8f5b-4264-9a88-a84a62b22081 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:41.867031 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac31e8-8f5b-4264-9a88-a84a62b22081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:41.867349 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac31e8-8f5b-4264-9a88-a84a62b22081 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:41.868011 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac31e8-8f5b-4264-9a88-a84a62b22081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:41.869657 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac31e8-8f5b-4264-9a88-a84a62b22081 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:41.871686 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:41.880976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1337/2713] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:41.885164 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61961d4e-2b09-4e0e-850d-0ca19cecbdb5 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:41.895152 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61961d4e-2b09-4e0e-850d-0ca19cecbdb5 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:41.899740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85962a0-2769-4c30-a1f9-8986fbae0b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:41.900372 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85962a0-2769-4c30-a1f9-8986fbae0b65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:41.900648 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85962a0-2769-4c30-a1f9-8986fbae0b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:41.900916 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85962a0-2769-4c30-a1f9-8986fbae0b65 admin admin] PATH_INFO: `/v3/projects/cbcb778f640c4d6b902a3782190e4fc6/users/7b8a0963c4504b198dd7e9da61cb387c/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:41.902765 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85962a0-2769-4c30-a1f9-8986fbae0b65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbcb778f640c4d6b902a3782190e4fc6, user_id=7b8a0963c4504b198dd7e9da61cb387c, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:41.905371 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85962a0-2769-4c30-a1f9-8986fbae0b65 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:41.915663 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1377/2714] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:41 2022] PUT /identity/v3/projects/cbcb778f640c4d6b902a3782190e4fc6/users/7b8a0963c4504b198dd7e9da61cb387c/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:41.927364 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adadbce-cd16-4c55-9241-cfa2320c2d71 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:41.927751 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adadbce-cd16-4c55-9241-cfa2320c2d71 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:41.928035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adadbce-cd16-4c55-9241-cfa2320c2d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:41.955895 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-0adadbce-cd16-4c55-9241-cfa2320c2d71 None None] MFA Rules not processed for user `7b8a0963c4504b198dd7e9da61cb387c`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:41.957108 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-0adadbce-cd16-4c55-9241-cfa2320c2d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:42.023587 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1338/2715] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:41 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:42.034606 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85962a0-2769-4c30-a1f9-8986fbae0b65 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:42.036422 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a85962a0-2769-4c30-a1f9-8986fbae0b65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:42.093524 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85962a0-2769-4c30-a1f9-8986fbae0b65 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:42.097158 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900c9d2-fa20-40fb-82b5-0ac7fd78689f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:42.097747 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900c9d2-fa20-40fb-82b5-0ac7fd78689f service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:42.098050 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900c9d2-fa20-40fb-82b5-0ac7fd78689f service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:42.098317 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900c9d2-fa20-40fb-82b5-0ac7fd78689f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:42.106991 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c900c9d2-fa20-40fb-82b5-0ac7fd78689f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:42.108434 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c900c9d2-fa20-40fb-82b5-0ac7fd78689f service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:42.118048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1378/2716] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:42 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:16:42.334543 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adadbce-cd16-4c55-9241-cfa2320c2d71 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:42.411098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adadbce-cd16-4c55-9241-cfa2320c2d71 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:42.416506 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e375ba-f2cf-47da-9e16-04ca77a51c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:42.417135 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e375ba-f2cf-47da-9e16-04ca77a51c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:42.417485 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e375ba-f2cf-47da-9e16-04ca77a51c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:42.417812 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e375ba-f2cf-47da-9e16-04ca77a51c5b admin admin] PATH_INFO: `/v3/users/5eb01b576f41426ab8c09dcdbbce4737` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:42.418983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e375ba-f2cf-47da-9e16-04ca77a51c5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eb01b576f41426ab8c09dcdbbce4737)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:42.421159 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e375ba-f2cf-47da-9e16-04ca77a51c5b admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:42.468900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-58e375ba-f2cf-47da-9e16-04ca77a51c5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eb01b576f41426ab8c09dcdbbce4737'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:42.469548 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-58e375ba-f2cf-47da-9e16-04ca77a51c5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eb01b576f41426ab8c09dcdbbce4737'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:42.481942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-58e375ba-f2cf-47da-9e16-04ca77a51c5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eb01b576f41426ab8c09dcdbbce4737'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:42.490961 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-58e375ba-f2cf-47da-9e16-04ca77a51c5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eb01b576f41426ab8c09dcdbbce4737'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:42.491804 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-58e375ba-f2cf-47da-9e16-04ca77a51c5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eb01b576f41426ab8c09dcdbbce4737'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:42.507829 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1339/2717] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:42 2022] DELETE /identity/v3/users/5eb01b576f41426ab8c09dcdbbce4737 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:42.513087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce603f-bc32-4e06-8cf3-a02e285f0726 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:42.513360 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce603f-bc32-4e06-8cf3-a02e285f0726 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:42.513703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ce603f-bc32-4e06-8cf3-a02e285f0726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:42.542357 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-e4ce603f-bc32-4e06-8cf3-a02e285f0726 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:42.543659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-e4ce603f-bc32-4e06-8cf3-a02e285f0726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:42.622745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1379/2718] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:42.633525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e375ba-f2cf-47da-9e16-04ca77a51c5b admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:42.635176 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-58e375ba-f2cf-47da-9e16-04ca77a51c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:42.641022 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:42.720631 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e375ba-f2cf-47da-9e16-04ca77a51c5b admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:42.724835 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f34b1-7512-402b-8b1b-906604774735 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:42.725628 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f34b1-7512-402b-8b1b-906604774735 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:42.725917 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f34b1-7512-402b-8b1b-906604774735 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:42.726183 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9f34b1-7512-402b-8b1b-906604774735 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:42.739173 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9f34b1-7512-402b-8b1b-906604774735 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:42.740730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9f34b1-7512-402b-8b1b-906604774735 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:42.761072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1340/2719] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:16:43.077171 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce603f-bc32-4e06-8cf3-a02e285f0726 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:43.078589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-e4ce603f-bc32-4e06-8cf3-a02e285f0726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:43.093407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ce603f-bc32-4e06-8cf3-a02e285f0726 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:43.099825 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173c70a-069c-4687-b528-d1abadf74344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:43.103541 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173c70a-069c-4687-b528-d1abadf74344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:43.103933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173c70a-069c-4687-b528-d1abadf74344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:43.104120 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173c70a-069c-4687-b528-d1abadf74344 admin admin] PATH_INFO: `/v3/projects/33d80eeac5254e048acbd6bbf362c52b` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:43.105354 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173c70a-069c-4687-b528-d1abadf74344 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33d80eeac5254e048acbd6bbf362c52b)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:43.107324 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173c70a-069c-4687-b528-d1abadf74344 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:43.158970 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-8173c70a-069c-4687-b528-d1abadf74344 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33d80eeac5254e048acbd6bbf362c52b', 'request_id': 'req-8173c70a-069c-4687-b528-d1abadf74344'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:43.166301 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-8173c70a-069c-4687-b528-d1abadf74344 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33d80eeac5254e048acbd6bbf362c52b', 'request_id': 'req-8173c70a-069c-4687-b528-d1abadf74344'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:43.172066 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-8173c70a-069c-4687-b528-d1abadf74344 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33d80eeac5254e048acbd6bbf362c52b', 'request_id': 'req-8173c70a-069c-4687-b528-d1abadf74344'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:43.178969 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:43.185046 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-8173c70a-069c-4687-b528-d1abadf74344 admin admin] The token cache is being invalidate because project 33d80eeac5254e048acbd6bbf362c52b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:43.185583 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-8173c70a-069c-4687-b528-d1abadf74344 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:43.186848 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-8173c70a-069c-4687-b528-d1abadf74344 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:43.188675 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1380/2720] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:43 2022] DELETE /identity/v3/projects/33d80eeac5254e048acbd6bbf362c52b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:43.193105 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f34b1-7512-402b-8b1b-906604774735 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:43.202626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9f34b1-7512-402b-8b1b-906604774735 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:43.261461 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c9ccd-c7fc-4402-8694-4958fe57560e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:43.262092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c9ccd-c7fc-4402-8694-4958fe57560e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:43.262375 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c9ccd-c7fc-4402-8694-4958fe57560e admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:43.262651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c9ccd-c7fc-4402-8694-4958fe57560e admin admin] PATH_INFO: `/v3/users/7b8a0963c4504b198dd7e9da61cb387c` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:43.263845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c9ccd-c7fc-4402-8694-4958fe57560e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b8a0963c4504b198dd7e9da61cb387c)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:43.266170 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c9ccd-c7fc-4402-8694-4958fe57560e admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:43.312782 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-be5c9ccd-c7fc-4402-8694-4958fe57560e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b8a0963c4504b198dd7e9da61cb387c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:43.313444 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-be5c9ccd-c7fc-4402-8694-4958fe57560e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b8a0963c4504b198dd7e9da61cb387c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:43.325137 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-be5c9ccd-c7fc-4402-8694-4958fe57560e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b8a0963c4504b198dd7e9da61cb387c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:43.332822 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-be5c9ccd-c7fc-4402-8694-4958fe57560e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b8a0963c4504b198dd7e9da61cb387c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:43.333773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-be5c9ccd-c7fc-4402-8694-4958fe57560e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b8a0963c4504b198dd7e9da61cb387c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:43.347647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1341/2721] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:43 2022] DELETE /identity/v3/users/7b8a0963c4504b198dd7e9da61cb387c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:43.623516 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173c70a-069c-4687-b528-d1abadf74344 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:43.624896 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8173c70a-069c-4687-b528-d1abadf74344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:43.638332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173c70a-069c-4687-b528-d1abadf74344 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:43.696903 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cec501-9744-42f6-af31-eecc69404493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:43.697528 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cec501-9744-42f6-af31-eecc69404493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:43.697819 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cec501-9744-42f6-af31-eecc69404493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:43.698157 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cec501-9744-42f6-af31-eecc69404493 admin admin] PATH_INFO: `/v3/projects/cbcb778f640c4d6b902a3782190e4fc6` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:43.699209 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cec501-9744-42f6-af31-eecc69404493 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbcb778f640c4d6b902a3782190e4fc6)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:43.701098 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cec501-9744-42f6-af31-eecc69404493 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:43.749954 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d1cec501-9744-42f6-af31-eecc69404493 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbcb778f640c4d6b902a3782190e4fc6', 'request_id': 'req-d1cec501-9744-42f6-af31-eecc69404493'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:43.757964 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d1cec501-9744-42f6-af31-eecc69404493 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbcb778f640c4d6b902a3782190e4fc6', 'request_id': 'req-d1cec501-9744-42f6-af31-eecc69404493'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:43.767649 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d1cec501-9744-42f6-af31-eecc69404493 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbcb778f640c4d6b902a3782190e4fc6', 'request_id': 'req-d1cec501-9744-42f6-af31-eecc69404493'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:43.781113 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d1cec501-9744-42f6-af31-eecc69404493 admin admin] The token cache is being invalidate because project cbcb778f640c4d6b902a3782190e4fc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:43.781524 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d1cec501-9744-42f6-af31-eecc69404493 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:43.782345 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d1cec501-9744-42f6-af31-eecc69404493 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:43.787390 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1381/2722] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:43 2022] DELETE /identity/v3/projects/cbcb778f640c4d6b902a3782190e4fc6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:16:43.790174 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22aa30b-0d20-4c17-8761-cdf0cd1de350 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:43.790174 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22aa30b-0d20-4c17-8761-cdf0cd1de350 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:43.790174 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22aa30b-0d20-4c17-8761-cdf0cd1de350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:43.809880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-f22aa30b-0d20-4c17-8761-cdf0cd1de350 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:43.811117 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-f22aa30b-0d20-4c17-8761-cdf0cd1de350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:43.832293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:43.876093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1342/2723] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:16:43.884253 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49597545-af60-48a9-b391-daa03ae2aeee None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:43.884546 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49597545-af60-48a9-b391-daa03ae2aeee None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:43.884847 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49597545-af60-48a9-b391-daa03ae2aeee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:43.905376 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-49597545-af60-48a9-b391-daa03ae2aeee None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:43.906570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-49597545-af60-48a9-b391-daa03ae2aeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:43.918696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1382/2724] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:43.920509 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22aa30b-0d20-4c17-8761-cdf0cd1de350 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:43.932091 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22aa30b-0d20-4c17-8761-cdf0cd1de350 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:43.938315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506899f9-957d-4063-850a-20d5a69dc849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:43.938958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506899f9-957d-4063-850a-20d5a69dc849 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:43.939303 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506899f9-957d-4063-850a-20d5a69dc849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:43.939579 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506899f9-957d-4063-850a-20d5a69dc849 admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:43.940763 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506899f9-957d-4063-850a-20d5a69dc849 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:16:43.941071 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506899f9-957d-4063-850a-20d5a69dc849 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:43.942773 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506899f9-957d-4063-850a-20d5a69dc849 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:43.956858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1343/2725] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:43.961645 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49597545-af60-48a9-b391-daa03ae2aeee None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:43.972837 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49597545-af60-48a9-b391-daa03ae2aeee None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:43.978842 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32274171-fa16-4bff-8ac1-d49b09c54e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:43.979589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32274171-fa16-4bff-8ac1-d49b09c54e10 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:43.979867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32274171-fa16-4bff-8ac1-d49b09c54e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:43.980134 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32274171-fa16-4bff-8ac1-d49b09c54e10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:43.980756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32274171-fa16-4bff-8ac1-d49b09c54e10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:43.982645 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32274171-fa16-4bff-8ac1-d49b09c54e10 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:44.030701 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1383/2726] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:16:43 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:44.031976 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506899f9-957d-4063-850a-20d5a69dc849 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:44.043039 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506899f9-957d-4063-850a-20d5a69dc849 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:44.100108 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa122cc5-1fa5-4de2-8cbc-39ce4816c10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:44.100717 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa122cc5-1fa5-4de2-8cbc-39ce4816c10a admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:44.101001 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa122cc5-1fa5-4de2-8cbc-39ce4816c10a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:44.101272 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa122cc5-1fa5-4de2-8cbc-39ce4816c10a admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:44.101895 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa122cc5-1fa5-4de2-8cbc-39ce4816c10a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:44.103905 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa122cc5-1fa5-4de2-8cbc-39ce4816c10a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:44.124992 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1344/2727] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:16:44 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:44.129867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32274171-fa16-4bff-8ac1-d49b09c54e10 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:44.140652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32274171-fa16-4bff-8ac1-d49b09c54e10 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:44.145599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc6b3d-75d3-4296-8b9d-87c645cdc9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:44.146270 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc6b3d-75d3-4296-8b9d-87c645cdc9ff admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:44.146553 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc6b3d-75d3-4296-8b9d-87c645cdc9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:44.146848 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fc6b3d-75d3-4296-8b9d-87c645cdc9ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:44.147619 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc6b3d-75d3-4296-8b9d-87c645cdc9ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:44.149258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fc6b3d-75d3-4296-8b9d-87c645cdc9ff admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:44.160472 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1384/2728] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:44.164602 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa122cc5-1fa5-4de2-8cbc-39ce4816c10a admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:44.174452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa122cc5-1fa5-4de2-8cbc-39ce4816c10a admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:44.179855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c13b08-89eb-4f48-ac30-945db9383457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:44.180467 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c13b08-89eb-4f48-ac30-945db9383457 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:44.180730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c13b08-89eb-4f48-ac30-945db9383457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:44.181004 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c13b08-89eb-4f48-ac30-945db9383457 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:44.181651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c13b08-89eb-4f48-ac30-945db9383457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:44.183272 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c13b08-89eb-4f48-ac30-945db9383457 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:44.195170 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1345/2729] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:44.199777 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc6b3d-75d3-4296-8b9d-87c645cdc9ff admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:44.216705 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fc6b3d-75d3-4296-8b9d-87c645cdc9ff admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:44.221378 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabed7cb-b2bc-4544-8d39-f183e0aeb25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:44.222027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabed7cb-b2bc-4544-8d39-f183e0aeb25a admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:44.222304 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabed7cb-b2bc-4544-8d39-f183e0aeb25a admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:44.222600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabed7cb-b2bc-4544-8d39-f183e0aeb25a admin admin] PATH_INFO: `/v3/projects/0ea3e378d2be4cdb9d9c175b1f08265c/users/d67a11a6bed74a9b94d88c641819c791/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:44.237650 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabed7cb-b2bc-4544-8d39-f183e0aeb25a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ea3e378d2be4cdb9d9c175b1f08265c, user_id=d67a11a6bed74a9b94d88c641819c791, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:44.240379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabed7cb-b2bc-4544-8d39-f183e0aeb25a admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:44.253894 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1385/2730] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:44 2022] PUT /identity/v3/projects/0ea3e378d2be4cdb9d9c175b1f08265c/users/d67a11a6bed74a9b94d88c641819c791/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:44.264471 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c228f0-3dbd-437a-816d-67ba52a45ed1 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:44.264740 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c228f0-3dbd-437a-816d-67ba52a45ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:44.265012 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c228f0-3dbd-437a-816d-67ba52a45ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:44.293339 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-77c228f0-3dbd-437a-816d-67ba52a45ed1 None None] MFA Rules not processed for user `d67a11a6bed74a9b94d88c641819c791`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:44.294613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-77c228f0-3dbd-437a-816d-67ba52a45ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:44.368856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1346/2731] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:44.373057 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabed7cb-b2bc-4544-8d39-f183e0aeb25a admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:44.411749 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:44.448885 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabed7cb-b2bc-4544-8d39-f183e0aeb25a admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:44.453939 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59805e0-883c-415f-9333-93560de3456e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:44.454544 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59805e0-883c-415f-9333-93560de3456e admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:44.454828 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59805e0-883c-415f-9333-93560de3456e admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:44.455130 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59805e0-883c-415f-9333-93560de3456e admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:44.455774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59805e0-883c-415f-9333-93560de3456e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:44.457703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59805e0-883c-415f-9333-93560de3456e admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:44.485055 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1386/2732] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:16:44 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:44.489719 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c228f0-3dbd-437a-816d-67ba52a45ed1 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:44.500608 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c228f0-3dbd-437a-816d-67ba52a45ed1 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:44.517328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:44.562381 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a56617-947b-4008-a52a-6d278ec3b58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:44.562992 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a56617-947b-4008-a52a-6d278ec3b58d admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:44.563293 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a56617-947b-4008-a52a-6d278ec3b58d admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:44.563559 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a56617-947b-4008-a52a-6d278ec3b58d admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:44.564186 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a56617-947b-4008-a52a-6d278ec3b58d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:44.566437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a56617-947b-4008-a52a-6d278ec3b58d admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:44.588109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1347/2733] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:16:44 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:16:44.592176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59805e0-883c-415f-9333-93560de3456e admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:44.603406 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59805e0-883c-415f-9333-93560de3456e admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:44.607971 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a1ae2-760f-4799-a2e6-3b743443589b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:44.608589 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a1ae2-760f-4799-a2e6-3b743443589b admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:44.608864 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a1ae2-760f-4799-a2e6-3b743443589b admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:44.609130 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a1ae2-760f-4799-a2e6-3b743443589b admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:44.609759 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a1ae2-760f-4799-a2e6-3b743443589b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:44.611386 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a1ae2-760f-4799-a2e6-3b743443589b admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:44.622328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1387/2734] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:44.626320 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a56617-947b-4008-a52a-6d278ec3b58d admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:44.636874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a56617-947b-4008-a52a-6d278ec3b58d admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:44.641654 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93379f30-c63c-479e-8856-dd12ebda4b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:44.642456 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93379f30-c63c-479e-8856-dd12ebda4b10 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:44.642663 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93379f30-c63c-479e-8856-dd12ebda4b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:44.642934 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93379f30-c63c-479e-8856-dd12ebda4b10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:44.643640 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93379f30-c63c-479e-8856-dd12ebda4b10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:44.645248 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93379f30-c63c-479e-8856-dd12ebda4b10 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:44.655868 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1348/2735] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:44.660746 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a1ae2-760f-4799-a2e6-3b743443589b admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:44.670977 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a1ae2-760f-4799-a2e6-3b743443589b admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:44.676248 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85d557-27ac-4899-8161-6b12e8cef957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:44.676894 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85d557-27ac-4899-8161-6b12e8cef957 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:44.677175 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85d557-27ac-4899-8161-6b12e8cef957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:44.677450 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85d557-27ac-4899-8161-6b12e8cef957 admin admin] PATH_INFO: `/v3/projects/d8cb0bd0dee54a44966dd48debfd175c/users/2393b6ef0d77416588ced814c4283008/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:44.692237 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85d557-27ac-4899-8161-6b12e8cef957 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8cb0bd0dee54a44966dd48debfd175c, user_id=2393b6ef0d77416588ced814c4283008, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:44.694938 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85d557-27ac-4899-8161-6b12e8cef957 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:44.707708 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1388/2736] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:44 2022] PUT /identity/v3/projects/d8cb0bd0dee54a44966dd48debfd175c/users/2393b6ef0d77416588ced814c4283008/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:44.719597 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fbba4-9326-4f76-bd9b-fc0fe907f7fb None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:44.719896 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fbba4-9326-4f76-bd9b-fc0fe907f7fb None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:44.720143 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fbba4-9326-4f76-bd9b-fc0fe907f7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:44.749771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-b38fbba4-9326-4f76-bd9b-fc0fe907f7fb None None] MFA Rules not processed for user `2393b6ef0d77416588ced814c4283008`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:44.750891 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-b38fbba4-9326-4f76-bd9b-fc0fe907f7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:44.816705 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1349/2737] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:44.822570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85d557-27ac-4899-8161-6b12e8cef957 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:44.886565 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85d557-27ac-4899-8161-6b12e8cef957 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:44.891402 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf58882-77fb-49a7-a35a-4c8f8f94debc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:44.892028 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf58882-77fb-49a7-a35a-4c8f8f94debc admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:44.892308 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf58882-77fb-49a7-a35a-4c8f8f94debc admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:44.892578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf58882-77fb-49a7-a35a-4c8f8f94debc admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:44.893194 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf58882-77fb-49a7-a35a-4c8f8f94debc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:44.895625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf58882-77fb-49a7-a35a-4c8f8f94debc admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:44.922579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1389/2738] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:16:44 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:44.926975 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fbba4-9326-4f76-bd9b-fc0fe907f7fb None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:44.943297 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fbba4-9326-4f76-bd9b-fc0fe907f7fb None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:45.002095 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c515b5-793d-4602-bc7e-0352f5ee9135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:45.002623 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c515b5-793d-4602-bc7e-0352f5ee9135 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:45.002920 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c515b5-793d-4602-bc7e-0352f5ee9135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:45.003229 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c515b5-793d-4602-bc7e-0352f5ee9135 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:45.003865 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c515b5-793d-4602-bc7e-0352f5ee9135 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:45.005774 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c515b5-793d-4602-bc7e-0352f5ee9135 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:45.028340 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1350/2739] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:16:44 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:16:45.033408 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf58882-77fb-49a7-a35a-4c8f8f94debc admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:45.048375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf58882-77fb-49a7-a35a-4c8f8f94debc admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:45.053287 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6be91-9bb6-4592-9ba1-845ec3b1056e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:45.053885 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6be91-9bb6-4592-9ba1-845ec3b1056e admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:45.054169 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6be91-9bb6-4592-9ba1-845ec3b1056e admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:45.054436 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6be91-9bb6-4592-9ba1-845ec3b1056e admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:45.055101 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6be91-9bb6-4592-9ba1-845ec3b1056e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:45.056945 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6be91-9bb6-4592-9ba1-845ec3b1056e admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:45.070567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1390/2740] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:45.074917 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c515b5-793d-4602-bc7e-0352f5ee9135 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:45.086356 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c515b5-793d-4602-bc7e-0352f5ee9135 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:45.092082 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f3097-98a4-4d33-9cb0-8f7674be443c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:45.093013 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f3097-98a4-4d33-9cb0-8f7674be443c admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:45.093294 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f3097-98a4-4d33-9cb0-8f7674be443c admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:45.093567 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f3097-98a4-4d33-9cb0-8f7674be443c admin admin] PATH_INFO: `/v3/projects/d24572ea4ec041af9ac6ae592fb6f15a/users/66d83af969344a1abc8863933fdb91ae/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:45.113383 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f3097-98a4-4d33-9cb0-8f7674be443c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d24572ea4ec041af9ac6ae592fb6f15a, user_id=66d83af969344a1abc8863933fdb91ae, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:45.113968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612f3097-98a4-4d33-9cb0-8f7674be443c admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:45.116263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:45.133848 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1351/2741] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:45 2022] PUT /identity/v3/projects/d24572ea4ec041af9ac6ae592fb6f15a/users/66d83af969344a1abc8863933fdb91ae/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:45.134866 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6be91-9bb6-4592-9ba1-845ec3b1056e admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:45.146518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6be91-9bb6-4592-9ba1-845ec3b1056e admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:45.191442 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:45.203600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33d83a-408f-446e-bbf6-3c6fa8e74cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:45.204226 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33d83a-408f-446e-bbf6-3c6fa8e74cbc admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:45.204504 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33d83a-408f-446e-bbf6-3c6fa8e74cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:45.204773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33d83a-408f-446e-bbf6-3c6fa8e74cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:45.205406 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33d83a-408f-446e-bbf6-3c6fa8e74cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:45.207643 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33d83a-408f-446e-bbf6-3c6fa8e74cbc admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:45.217405 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1391/2742] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:16:45.221712 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f3097-98a4-4d33-9cb0-8f7674be443c admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:45.232112 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612f3097-98a4-4d33-9cb0-8f7674be443c admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:45.236890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f529a8-28b8-494c-88c6-c4203197cbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:45.237529 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f529a8-28b8-494c-88c6-c4203197cbe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:45.237809 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f529a8-28b8-494c-88c6-c4203197cbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:45.238081 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f529a8-28b8-494c-88c6-c4203197cbe7 admin admin] PATH_INFO: `/v3/projects/d24572ea4ec041af9ac6ae592fb6f15a/users/66d83af969344a1abc8863933fdb91ae/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:45.240029 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f529a8-28b8-494c-88c6-c4203197cbe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d24572ea4ec041af9ac6ae592fb6f15a, user_id=66d83af969344a1abc8863933fdb91ae, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:45.242925 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f529a8-28b8-494c-88c6-c4203197cbe7 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:45.254107 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1352/2743] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:45 2022] PUT /identity/v3/projects/d24572ea4ec041af9ac6ae592fb6f15a/users/66d83af969344a1abc8863933fdb91ae/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:45.264147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32764fa1-807b-4dd8-a12c-3174bf0ebff7 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:45.264451 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32764fa1-807b-4dd8-a12c-3174bf0ebff7 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:45.264721 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32764fa1-807b-4dd8-a12c-3174bf0ebff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:45.293265 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-32764fa1-807b-4dd8-a12c-3174bf0ebff7 None None] MFA Rules not processed for user `66d83af969344a1abc8863933fdb91ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:45.294441 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-32764fa1-807b-4dd8-a12c-3174bf0ebff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:45.364747 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1392/2744] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:45.375941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f529a8-28b8-494c-88c6-c4203197cbe7 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:45.378317 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-a5f529a8-28b8-494c-88c6-c4203197cbe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:45.438451 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f529a8-28b8-494c-88c6-c4203197cbe7 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:45.442720 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17a044-246f-4501-9ef7-21524732c53f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:45.443355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17a044-246f-4501-9ef7-21524732c53f service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:45.443659 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17a044-246f-4501-9ef7-21524732c53f service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:45.443931 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17a044-246f-4501-9ef7-21524732c53f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:45.453217 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17a044-246f-4501-9ef7-21524732c53f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:45.454699 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17a044-246f-4501-9ef7-21524732c53f service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:45.465093 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1353/2745] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:45 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:16:45.506311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32764fa1-807b-4dd8-a12c-3174bf0ebff7 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:45.507669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-32764fa1-807b-4dd8-a12c-3174bf0ebff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:45.520127 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32764fa1-807b-4dd8-a12c-3174bf0ebff7 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:45.523755 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d4aa1a-81c5-4306-81b7-4ee206b64ff2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:45.524380 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4aa1a-81c5-4306-81b7-4ee206b64ff2 service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:45.524658 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4aa1a-81c5-4306-81b7-4ee206b64ff2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:45.524927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d4aa1a-81c5-4306-81b7-4ee206b64ff2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:45.586734 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4aa1a-81c5-4306-81b7-4ee206b64ff2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:45.588193 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d4aa1a-81c5-4306-81b7-4ee206b64ff2 service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:45.597892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1393/2746] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:16:47.324941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17a044-246f-4501-9ef7-21524732c53f service cinder] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:47.326347 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-8d17a044-246f-4501-9ef7-21524732c53f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:47.404609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17a044-246f-4501-9ef7-21524732c53f service cinder] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:47.409850 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db01910-a30e-43c4-8fa2-5c6fff7e3257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:47.410505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db01910-a30e-43c4-8fa2-5c6fff7e3257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:47.410784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db01910-a30e-43c4-8fa2-5c6fff7e3257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:47.411098 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db01910-a30e-43c4-8fa2-5c6fff7e3257 admin admin] PATH_INFO: `/v3/users/3e2f32e5ca814d389077251a57cd41ce` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:47.412212 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db01910-a30e-43c4-8fa2-5c6fff7e3257 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e2f32e5ca814d389077251a57cd41ce)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:47.414102 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db01910-a30e-43c4-8fa2-5c6fff7e3257 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:47.465274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-8db01910-a30e-43c4-8fa2-5c6fff7e3257 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e2f32e5ca814d389077251a57cd41ce'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:47.465935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-8db01910-a30e-43c4-8fa2-5c6fff7e3257 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e2f32e5ca814d389077251a57cd41ce'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:47.477970 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-8db01910-a30e-43c4-8fa2-5c6fff7e3257 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e2f32e5ca814d389077251a57cd41ce'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:47.484605 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-8db01910-a30e-43c4-8fa2-5c6fff7e3257 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e2f32e5ca814d389077251a57cd41ce'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:47.485239 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-8db01910-a30e-43c4-8fa2-5c6fff7e3257 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e2f32e5ca814d389077251a57cd41ce'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:47.498581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1354/2747] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:47 2022] DELETE /identity/v3/users/3e2f32e5ca814d389077251a57cd41ce => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:47.503701 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349ebae-74e7-40d5-8b73-193f1c597cd2 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:47.504203 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349ebae-74e7-40d5-8b73-193f1c597cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:47.504306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6349ebae-74e7-40d5-8b73-193f1c597cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:47.523400 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-6349ebae-74e7-40d5-8b73-193f1c597cd2 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:47.524572 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6349ebae-74e7-40d5-8b73-193f1c597cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:47.583686 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1394/2748] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:47.593416 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db01910-a30e-43c4-8fa2-5c6fff7e3257 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:47.594980 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-8db01910-a30e-43c4-8fa2-5c6fff7e3257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:47.601873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:47.661218 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db01910-a30e-43c4-8fa2-5c6fff7e3257 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:47.665072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b9db3-0e2d-45fe-9e23-369ca4a8830d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:47.665637 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b9db3-0e2d-45fe-9e23-369ca4a8830d service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:47.665945 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b9db3-0e2d-45fe-9e23-369ca4a8830d service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:47.666204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b9db3-0e2d-45fe-9e23-369ca4a8830d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:47.675525 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b9db3-0e2d-45fe-9e23-369ca4a8830d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:47.676940 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b9db3-0e2d-45fe-9e23-369ca4a8830d service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:47.686128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1355/2749] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:16:47.970532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349ebae-74e7-40d5-8b73-193f1c597cd2 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:47.972042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6349ebae-74e7-40d5-8b73-193f1c597cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:47.988368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6349ebae-74e7-40d5-8b73-193f1c597cd2 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:47.994548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad1588-6c30-4f31-8b39-6ecd0a972866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:47.995569 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad1588-6c30-4f31-8b39-6ecd0a972866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:47.996199 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad1588-6c30-4f31-8b39-6ecd0a972866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:47.996199 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad1588-6c30-4f31-8b39-6ecd0a972866 admin admin] PATH_INFO: `/v3/projects/738ce66d0d5c46d6a1c7829a4c941fad` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:47.997854 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad1588-6c30-4f31-8b39-6ecd0a972866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=738ce66d0d5c46d6a1c7829a4c941fad)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:47.999870 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad1588-6c30-4f31-8b39-6ecd0a972866 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:48.008922 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:48.061638 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d8ad1588-6c30-4f31-8b39-6ecd0a972866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '738ce66d0d5c46d6a1c7829a4c941fad', 'request_id': 'req-d8ad1588-6c30-4f31-8b39-6ecd0a972866'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:48.069440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d8ad1588-6c30-4f31-8b39-6ecd0a972866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '738ce66d0d5c46d6a1c7829a4c941fad', 'request_id': 'req-d8ad1588-6c30-4f31-8b39-6ecd0a972866'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:48.075248 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d8ad1588-6c30-4f31-8b39-6ecd0a972866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '738ce66d0d5c46d6a1c7829a4c941fad', 'request_id': 'req-d8ad1588-6c30-4f31-8b39-6ecd0a972866'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:48.086484 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d8ad1588-6c30-4f31-8b39-6ecd0a972866 admin admin] The token cache is being invalidate because project 738ce66d0d5c46d6a1c7829a4c941fad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:48.086770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d8ad1588-6c30-4f31-8b39-6ecd0a972866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:48.087438 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-d8ad1588-6c30-4f31-8b39-6ecd0a972866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:48.088899 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1395/2750] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:47 2022] DELETE /identity/v3/projects/738ce66d0d5c46d6a1c7829a4c941fad => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:48.093990 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f8c17-2cba-48c4-8947-571b81ec64fa None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:48.094281 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f8c17-2cba-48c4-8947-571b81ec64fa None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:48.094563 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f8c17-2cba-48c4-8947-571b81ec64fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:48.114204 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-a42f8c17-2cba-48c4-8947-571b81ec64fa None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:48.115416 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-a42f8c17-2cba-48c4-8947-571b81ec64fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:48.175882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1356/2751] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:16:48.184851 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e69ee9-4773-4c0f-9ee0-cb6b7aed09d5 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:48.185135 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e69ee9-4773-4c0f-9ee0-cb6b7aed09d5 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:48.185409 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e69ee9-4773-4c0f-9ee0-cb6b7aed09d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:48.208444 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-b5e69ee9-4773-4c0f-9ee0-cb6b7aed09d5 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:48.209569 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-b5e69ee9-4773-4c0f-9ee0-cb6b7aed09d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:48.217560 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1396/2752] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:48.222485 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f8c17-2cba-48c4-8947-571b81ec64fa None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:48.232300 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f8c17-2cba-48c4-8947-571b81ec64fa None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:48.238018 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456e44d-dfb3-4911-9711-a8e232563fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:48.238196 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456e44d-dfb3-4911-9711-a8e232563fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:48.238426 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456e44d-dfb3-4911-9711-a8e232563fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:48.238695 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456e44d-dfb3-4911-9711-a8e232563fd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:48.239468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456e44d-dfb3-4911-9711-a8e232563fd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:16:48.239849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456e44d-dfb3-4911-9711-a8e232563fd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:48.241444 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456e44d-dfb3-4911-9711-a8e232563fd3 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:48.253264 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1357/2753] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:16:48.257578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e69ee9-4773-4c0f-9ee0-cb6b7aed09d5 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:48.266789 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e69ee9-4773-4c0f-9ee0-cb6b7aed09d5 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:48.271507 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a41171-d50c-49fe-a048-5bd1b5ed4583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:48.272110 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a41171-d50c-49fe-a048-5bd1b5ed4583 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:48.272428 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a41171-d50c-49fe-a048-5bd1b5ed4583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:48.272699 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a41171-d50c-49fe-a048-5bd1b5ed4583 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:48.273336 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a41171-d50c-49fe-a048-5bd1b5ed4583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:48.275259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a41171-d50c-49fe-a048-5bd1b5ed4583 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:48.306389 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1397/2754] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:16:48 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:48.308064 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456e44d-dfb3-4911-9711-a8e232563fd3 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:48.318065 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456e44d-dfb3-4911-9711-a8e232563fd3 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:48.376415 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f5541-2d6e-436e-86a5-5bb7a20a20a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:48.377100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f5541-2d6e-436e-86a5-5bb7a20a20a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:48.377300 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f5541-2d6e-436e-86a5-5bb7a20a20a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:48.377572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f5541-2d6e-436e-86a5-5bb7a20a20a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:48.378220 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f5541-2d6e-436e-86a5-5bb7a20a20a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:48.380212 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f5541-2d6e-436e-86a5-5bb7a20a20a2 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:48.401865 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1358/2755] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:16:48 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:48.406213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a41171-d50c-49fe-a048-5bd1b5ed4583 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:48.417393 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a41171-d50c-49fe-a048-5bd1b5ed4583 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:48.422027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff9102-7464-4adb-aa22-3a74872c0cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:48.434307 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff9102-7464-4adb-aa22-3a74872c0cdc admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:48.434858 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff9102-7464-4adb-aa22-3a74872c0cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:48.435164 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff9102-7464-4adb-aa22-3a74872c0cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:48.435838 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff9102-7464-4adb-aa22-3a74872c0cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:48.437472 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff9102-7464-4adb-aa22-3a74872c0cdc admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:48.455747 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1398/2756] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:48.457101 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f5541-2d6e-436e-86a5-5bb7a20a20a2 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:48.467718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f5541-2d6e-436e-86a5-5bb7a20a20a2 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:48.472965 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbc7ad-fa95-40cd-ae8f-d9e753293075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:48.473572 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbc7ad-fa95-40cd-ae8f-d9e753293075 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:48.473853 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbc7ad-fa95-40cd-ae8f-d9e753293075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:48.474154 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fbc7ad-fa95-40cd-ae8f-d9e753293075 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:48.474763 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fbc7ad-fa95-40cd-ae8f-d9e753293075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:48.476392 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fbc7ad-fa95-40cd-ae8f-d9e753293075 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:48.495695 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1359/2757] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:48.496984 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff9102-7464-4adb-aa22-3a74872c0cdc admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:48.508505 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff9102-7464-4adb-aa22-3a74872c0cdc admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:48.514944 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b1cd1-67c4-401b-896d-b895adcde22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:48.515377 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbc7ad-fa95-40cd-ae8f-d9e753293075 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:48.515650 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b1cd1-67c4-401b-896d-b895adcde22c admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:48.515932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b1cd1-67c4-401b-896d-b895adcde22c admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:48.516280 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b1cd1-67c4-401b-896d-b895adcde22c admin admin] PATH_INFO: `/v3/projects/6f7eb94d4f644255a15f6afbe20b64b1/users/6b397b9393814b88bd78b45eef386573/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:48.517167 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-10fbc7ad-fa95-40cd-ae8f-d9e753293075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:48.530177 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fbc7ad-fa95-40cd-ae8f-d9e753293075 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:48.531692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b1cd1-67c4-401b-896d-b895adcde22c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f7eb94d4f644255a15f6afbe20b64b1, user_id=6b397b9393814b88bd78b45eef386573, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:48.534924 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b1cd1-67c4-401b-896d-b895adcde22c admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:48.535449 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23473464-d7ed-4b1b-b86c-a5b48f25a406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:48.536362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23473464-d7ed-4b1b-b86c-a5b48f25a406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:48.536649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23473464-d7ed-4b1b-b86c-a5b48f25a406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:48.536923 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23473464-d7ed-4b1b-b86c-a5b48f25a406 admin admin] PATH_INFO: `/v3/users/d67a11a6bed74a9b94d88c641819c791` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:48.538157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23473464-d7ed-4b1b-b86c-a5b48f25a406 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d67a11a6bed74a9b94d88c641819c791)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:48.540120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23473464-d7ed-4b1b-b86c-a5b48f25a406 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:48.546164 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:48.551951 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1399/2758] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:48 2022] PUT /identity/v3/projects/6f7eb94d4f644255a15f6afbe20b64b1/users/6b397b9393814b88bd78b45eef386573/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:48.563437 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e7f93-b37e-4d10-bbba-21216bebbe5c None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:48.563628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e7f93-b37e-4d10-bbba-21216bebbe5c None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:48.563859 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e7f93-b37e-4d10-bbba-21216bebbe5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:48.593411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-23473464-d7ed-4b1b-b86c-a5b48f25a406 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd67a11a6bed74a9b94d88c641819c791'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:48.593636 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-850e7f93-b37e-4d10-bbba-21216bebbe5c None None] MFA Rules not processed for user `6b397b9393814b88bd78b45eef386573`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:48.594117 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-23473464-d7ed-4b1b-b86c-a5b48f25a406 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd67a11a6bed74a9b94d88c641819c791'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:48.594613 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-850e7f93-b37e-4d10-bbba-21216bebbe5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:48.607197 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-23473464-d7ed-4b1b-b86c-a5b48f25a406 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd67a11a6bed74a9b94d88c641819c791'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:48.614491 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-23473464-d7ed-4b1b-b86c-a5b48f25a406 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd67a11a6bed74a9b94d88c641819c791'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:48.615374 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-23473464-d7ed-4b1b-b86c-a5b48f25a406 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd67a11a6bed74a9b94d88c641819c791'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:48.628844 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1360/2759] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:48 2022] DELETE /identity/v3/users/d67a11a6bed74a9b94d88c641819c791 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:16:48.633150 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c231890-1054-41fa-9917-36dc0183a9ec None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:48.633440 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c231890-1054-41fa-9917-36dc0183a9ec None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:48.633716 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c231890-1054-41fa-9917-36dc0183a9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:48.654977 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-6c231890-1054-41fa-9917-36dc0183a9ec None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:48.656328 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6c231890-1054-41fa-9917-36dc0183a9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:48.666435 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1400/2760] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:48 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:16:48.678497 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e7f93-b37e-4d10-bbba-21216bebbe5c None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:48.679937 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-850e7f93-b37e-4d10-bbba-21216bebbe5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:48.699673 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:48.720856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1361/2761] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:16:48.732334 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c231890-1054-41fa-9917-36dc0183a9ec None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:48.733755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6c231890-1054-41fa-9917-36dc0183a9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:48.741737 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e7f93-b37e-4d10-bbba-21216bebbe5c None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:48.794844 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abec3d-7625-4024-8360-4ed4c6943de1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:48.795636 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abec3d-7625-4024-8360-4ed4c6943de1 service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:48.795778 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c231890-1054-41fa-9917-36dc0183a9ec None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:48.796090 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abec3d-7625-4024-8360-4ed4c6943de1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:48.796451 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abec3d-7625-4024-8360-4ed4c6943de1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:48.799337 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8705e17-e584-4a9a-bb00-f890b5fe910b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:48.799970 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8705e17-e584-4a9a-bb00-f890b5fe910b service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:48.800252 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8705e17-e584-4a9a-bb00-f890b5fe910b service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:48.800532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8705e17-e584-4a9a-bb00-f890b5fe910b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:48.809934 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8705e17-e584-4a9a-bb00-f890b5fe910b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:48.812363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8705e17-e584-4a9a-bb00-f890b5fe910b service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:48.822472 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1362/2762] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:16:48.858534 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1abec3d-7625-4024-8360-4ed4c6943de1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:48.860774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1abec3d-7625-4024-8360-4ed4c6943de1 service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:48.870935 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1401/2763] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:48 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:16:49.135695 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8705e17-e584-4a9a-bb00-f890b5fe910b service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:49.137315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-e8705e17-e584-4a9a-bb00-f890b5fe910b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:49.174920 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8705e17-e584-4a9a-bb00-f890b5fe910b service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:49.180099 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1324c966-6a02-4dce-af3e-5e09ce5b821b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:49.180730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324c966-6a02-4dce-af3e-5e09ce5b821b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:49.181048 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324c966-6a02-4dce-af3e-5e09ce5b821b admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:49.181270 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324c966-6a02-4dce-af3e-5e09ce5b821b admin admin] PATH_INFO: `/v3/projects/0ea3e378d2be4cdb9d9c175b1f08265c` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:49.182464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1324c966-6a02-4dce-af3e-5e09ce5b821b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ea3e378d2be4cdb9d9c175b1f08265c)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:49.184440 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1324c966-6a02-4dce-af3e-5e09ce5b821b admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:49.213241 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:49.245123 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-1324c966-6a02-4dce-af3e-5e09ce5b821b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ea3e378d2be4cdb9d9c175b1f08265c', 'request_id': 'req-1324c966-6a02-4dce-af3e-5e09ce5b821b'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:49.251920 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-1324c966-6a02-4dce-af3e-5e09ce5b821b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ea3e378d2be4cdb9d9c175b1f08265c', 'request_id': 'req-1324c966-6a02-4dce-af3e-5e09ce5b821b'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:49.258511 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-1324c966-6a02-4dce-af3e-5e09ce5b821b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ea3e378d2be4cdb9d9c175b1f08265c', 'request_id': 'req-1324c966-6a02-4dce-af3e-5e09ce5b821b'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:49.271053 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-1324c966-6a02-4dce-af3e-5e09ce5b821b admin admin] The token cache is being invalidate because project 0ea3e378d2be4cdb9d9c175b1f08265c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:49.271325 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-1324c966-6a02-4dce-af3e-5e09ce5b821b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:49.271936 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-1324c966-6a02-4dce-af3e-5e09ce5b821b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:49.273880 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1363/2764] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:49 2022] DELETE /identity/v3/projects/0ea3e378d2be4cdb9d9c175b1f08265c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:49.277941 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abec3d-7625-4024-8360-4ed4c6943de1 service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:49.288570 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abec3d-7625-4024-8360-4ed4c6943de1 service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:49.345656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab19c5c-3a29-4147-b12d-ad7d4a7bc691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:49.346290 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab19c5c-3a29-4147-b12d-ad7d4a7bc691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:49.346568 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab19c5c-3a29-4147-b12d-ad7d4a7bc691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:49.346841 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab19c5c-3a29-4147-b12d-ad7d4a7bc691 admin admin] PATH_INFO: `/v3/users/2393b6ef0d77416588ced814c4283008` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:49.348020 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab19c5c-3a29-4147-b12d-ad7d4a7bc691 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2393b6ef0d77416588ced814c4283008)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:49.349925 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab19c5c-3a29-4147-b12d-ad7d4a7bc691 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:49.383937 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:49.397152 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-cab19c5c-3a29-4147-b12d-ad7d4a7bc691 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2393b6ef0d77416588ced814c4283008'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:49.403843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-cab19c5c-3a29-4147-b12d-ad7d4a7bc691 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2393b6ef0d77416588ced814c4283008'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:49.411001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-cab19c5c-3a29-4147-b12d-ad7d4a7bc691 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2393b6ef0d77416588ced814c4283008'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:49.411766 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-cab19c5c-3a29-4147-b12d-ad7d4a7bc691 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2393b6ef0d77416588ced814c4283008'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:49.412388 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-cab19c5c-3a29-4147-b12d-ad7d4a7bc691 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2393b6ef0d77416588ced814c4283008'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:49.429848 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1402/2765] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:49 2022] DELETE /identity/v3/users/2393b6ef0d77416588ced814c4283008 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:49.705950 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1324c966-6a02-4dce-af3e-5e09ce5b821b admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:49.707534 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1324c966-6a02-4dce-af3e-5e09ce5b821b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:49.722534 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1324c966-6a02-4dce-af3e-5e09ce5b821b admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:49.783367 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d4caa-305a-4bb5-be7d-d28c233e3920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:49.784686 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d4caa-305a-4bb5-be7d-d28c233e3920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:49.785082 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d4caa-305a-4bb5-be7d-d28c233e3920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:49.785357 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d4caa-305a-4bb5-be7d-d28c233e3920 admin admin] PATH_INFO: `/v3/projects/d8cb0bd0dee54a44966dd48debfd175c` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:49.786647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d4caa-305a-4bb5-be7d-d28c233e3920 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8cb0bd0dee54a44966dd48debfd175c)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:49.788581 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d4caa-305a-4bb5-be7d-d28c233e3920 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:49.835779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9a7d4caa-305a-4bb5-be7d-d28c233e3920 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8cb0bd0dee54a44966dd48debfd175c', 'request_id': 'req-9a7d4caa-305a-4bb5-be7d-d28c233e3920'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:49.841830 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9a7d4caa-305a-4bb5-be7d-d28c233e3920 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8cb0bd0dee54a44966dd48debfd175c', 'request_id': 'req-9a7d4caa-305a-4bb5-be7d-d28c233e3920'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:49.847986 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9a7d4caa-305a-4bb5-be7d-d28c233e3920 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8cb0bd0dee54a44966dd48debfd175c', 'request_id': 'req-9a7d4caa-305a-4bb5-be7d-d28c233e3920'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:49.860068 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9a7d4caa-305a-4bb5-be7d-d28c233e3920 admin admin] The token cache is being invalidate because project d8cb0bd0dee54a44966dd48debfd175c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:49.860362 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9a7d4caa-305a-4bb5-be7d-d28c233e3920 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:49.861013 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-9a7d4caa-305a-4bb5-be7d-d28c233e3920 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:49.863206 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1364/2766] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:49 2022] DELETE /identity/v3/projects/d8cb0bd0dee54a44966dd48debfd175c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:49.867154 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab19c5c-3a29-4147-b12d-ad7d4a7bc691 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:49.868578 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-cab19c5c-3a29-4147-b12d-ad7d4a7bc691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:49.881216 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab19c5c-3a29-4147-b12d-ad7d4a7bc691 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:49.937891 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4008126d-b6dd-45d8-8626-fbf7f810b4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:49.938521 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008126d-b6dd-45d8-8626-fbf7f810b4bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:49.938798 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008126d-b6dd-45d8-8626-fbf7f810b4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:49.939088 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4008126d-b6dd-45d8-8626-fbf7f810b4bf admin admin] PATH_INFO: `/v3/users/66d83af969344a1abc8863933fdb91ae` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:49.940220 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4008126d-b6dd-45d8-8626-fbf7f810b4bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=66d83af969344a1abc8863933fdb91ae)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:49.942130 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4008126d-b6dd-45d8-8626-fbf7f810b4bf admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:49.991518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-4008126d-b6dd-45d8-8626-fbf7f810b4bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66d83af969344a1abc8863933fdb91ae'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:49.998608 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-4008126d-b6dd-45d8-8626-fbf7f810b4bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66d83af969344a1abc8863933fdb91ae'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:50.007796 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-4008126d-b6dd-45d8-8626-fbf7f810b4bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66d83af969344a1abc8863933fdb91ae'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:50.008488 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-4008126d-b6dd-45d8-8626-fbf7f810b4bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66d83af969344a1abc8863933fdb91ae'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:50.009126 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-4008126d-b6dd-45d8-8626-fbf7f810b4bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66d83af969344a1abc8863933fdb91ae'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:50.026597 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1403/2767] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:16:49 2022] DELETE /identity/v3/users/66d83af969344a1abc8863933fdb91ae => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:50.327599 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d4caa-305a-4bb5-be7d-d28c233e3920 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:50.329539 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-9a7d4caa-305a-4bb5-be7d-d28c233e3920 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:50.354845 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d4caa-305a-4bb5-be7d-d28c233e3920 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:50.417538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:50.442215 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99239f-801d-4532-8602-f0b583057f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:50.442876 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99239f-801d-4532-8602-f0b583057f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:50.443194 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99239f-801d-4532-8602-f0b583057f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:50.443520 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99239f-801d-4532-8602-f0b583057f43 admin admin] PATH_INFO: `/v3/projects/d24572ea4ec041af9ac6ae592fb6f15a` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:50.444771 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99239f-801d-4532-8602-f0b583057f43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d24572ea4ec041af9ac6ae592fb6f15a)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:50.447197 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99239f-801d-4532-8602-f0b583057f43 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:50.507759 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7c99239f-801d-4532-8602-f0b583057f43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd24572ea4ec041af9ac6ae592fb6f15a', 'request_id': 'req-7c99239f-801d-4532-8602-f0b583057f43'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:50.514276 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7c99239f-801d-4532-8602-f0b583057f43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd24572ea4ec041af9ac6ae592fb6f15a', 'request_id': 'req-7c99239f-801d-4532-8602-f0b583057f43'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:50.520266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7c99239f-801d-4532-8602-f0b583057f43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd24572ea4ec041af9ac6ae592fb6f15a', 'request_id': 'req-7c99239f-801d-4532-8602-f0b583057f43'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:50.533386 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7c99239f-801d-4532-8602-f0b583057f43 admin admin] The token cache is being invalidate because project d24572ea4ec041af9ac6ae592fb6f15a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:16:50.533675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7c99239f-801d-4532-8602-f0b583057f43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:50.534801 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-7c99239f-801d-4532-8602-f0b583057f43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:16:50.536311 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1365/2768] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:16:50 2022] DELETE /identity/v3/projects/d24572ea4ec041af9ac6ae592fb6f15a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:16:50.542268 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64ed20-12d1-4308-bfa6-fcf155eca13d None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:50.542557 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64ed20-12d1-4308-bfa6-fcf155eca13d None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:50.542830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64ed20-12d1-4308-bfa6-fcf155eca13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:50.562842 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-6d64ed20-12d1-4308-bfa6-fcf155eca13d None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:50.563969 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6d64ed20-12d1-4308-bfa6-fcf155eca13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:50.585009 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:50.625773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1404/2769] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:50.635583 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7aef9-242b-4732-b598-c80794ab6aee None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:50.635870 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7aef9-242b-4732-b598-c80794ab6aee None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:50.636147 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7aef9-242b-4732-b598-c80794ab6aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:50.656431 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-35c7aef9-242b-4732-b598-c80794ab6aee None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:50.657631 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-35c7aef9-242b-4732-b598-c80794ab6aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:50.666370 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1366/2770] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:50.670745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64ed20-12d1-4308-bfa6-fcf155eca13d None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:50.681045 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64ed20-12d1-4308-bfa6-fcf155eca13d None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:50.685592 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64ef83-8fb4-4101-9781-040c5d2642cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:50.686178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64ef83-8fb4-4101-9781-040c5d2642cd admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:50.686457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64ef83-8fb4-4101-9781-040c5d2642cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:50.686724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64ef83-8fb4-4101-9781-040c5d2642cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:50.687396 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64ef83-8fb4-4101-9781-040c5d2642cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:16:50.687693 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64ef83-8fb4-4101-9781-040c5d2642cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:50.689270 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64ef83-8fb4-4101-9781-040c5d2642cd admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:50.700942 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1405/2771] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:16:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:50.705528 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7aef9-242b-4732-b598-c80794ab6aee None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:50.715171 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7aef9-242b-4732-b598-c80794ab6aee None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:50.719859 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3120dd-d752-4ad9-85ad-86d4f92d5887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:50.720453 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3120dd-d752-4ad9-85ad-86d4f92d5887 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:50.720734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3120dd-d752-4ad9-85ad-86d4f92d5887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:50.721005 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3120dd-d752-4ad9-85ad-86d4f92d5887 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:50.722119 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3120dd-d752-4ad9-85ad-86d4f92d5887 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:50.724222 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3120dd-d752-4ad9-85ad-86d4f92d5887 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:50.752179 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1367/2772] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:16:50 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:50.756429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64ef83-8fb4-4101-9781-040c5d2642cd admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:50.766390 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64ef83-8fb4-4101-9781-040c5d2642cd admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:50.822116 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca1caa-9fb9-445e-a28f-d6d528a04943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:50.822629 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca1caa-9fb9-445e-a28f-d6d528a04943 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:50.822908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca1caa-9fb9-445e-a28f-d6d528a04943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:50.823224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca1caa-9fb9-445e-a28f-d6d528a04943 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:50.823876 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca1caa-9fb9-445e-a28f-d6d528a04943 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:50.825778 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca1caa-9fb9-445e-a28f-d6d528a04943 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:50.850993 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1406/2773] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:16:50 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:50.853207 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3120dd-d752-4ad9-85ad-86d4f92d5887 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:50.864631 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3120dd-d752-4ad9-85ad-86d4f92d5887 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:50.869307 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7a593-6a0f-474b-b938-ec544538265a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:50.869914 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7a593-6a0f-474b-b938-ec544538265a admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:50.870190 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7a593-6a0f-474b-b938-ec544538265a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:50.870457 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7a593-6a0f-474b-b938-ec544538265a admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:50.871119 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b7a593-6a0f-474b-b938-ec544538265a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:50.872707 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b7a593-6a0f-474b-b938-ec544538265a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:50.883480 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1368/2774] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:16:50.887476 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca1caa-9fb9-445e-a28f-d6d528a04943 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:50.897773 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca1caa-9fb9-445e-a28f-d6d528a04943 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:50.902438 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d810b8-ead8-413a-82c7-93e0f0709d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:50.903042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d810b8-ead8-413a-82c7-93e0f0709d05 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:50.903354 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d810b8-ead8-413a-82c7-93e0f0709d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:50.903628 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d810b8-ead8-413a-82c7-93e0f0709d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:50.904269 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d810b8-ead8-413a-82c7-93e0f0709d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:50.905867 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d810b8-ead8-413a-82c7-93e0f0709d05 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:50.916934 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1407/2775] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:50.921075 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7a593-6a0f-474b-b938-ec544538265a admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:50.931616 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7a593-6a0f-474b-b938-ec544538265a admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:50.936726 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f52b8-dd21-4ae6-bff2-0bb078738040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:50.937403 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f52b8-dd21-4ae6-bff2-0bb078738040 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:50.937687 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f52b8-dd21-4ae6-bff2-0bb078738040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:50.938028 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f52b8-dd21-4ae6-bff2-0bb078738040 admin admin] PATH_INFO: `/v3/projects/09cf446b5e344735846f09b4de104575/users/25611f981ea74fe595c655a01f4aadcc/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:50.952748 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f52b8-dd21-4ae6-bff2-0bb078738040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09cf446b5e344735846f09b4de104575, user_id=25611f981ea74fe595c655a01f4aadcc, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:50.955399 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f52b8-dd21-4ae6-bff2-0bb078738040 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:50.970132 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1369/2776] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:50 2022] PUT /identity/v3/projects/09cf446b5e344735846f09b4de104575/users/25611f981ea74fe595c655a01f4aadcc/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:50.975213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11b716-d634-4095-9c09-05a00831d6b3 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:50.975498 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11b716-d634-4095-9c09-05a00831d6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:50.975771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11b716-d634-4095-9c09-05a00831d6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:50.994381 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-aa11b716-d634-4095-9c09-05a00831d6b3 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:50.995646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-aa11b716-d634-4095-9c09-05a00831d6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:51.054084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1408/2777] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:51.064296 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f52b8-dd21-4ae6-bff2-0bb078738040 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:51.066081 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ad3f52b8-dd21-4ae6-bff2-0bb078738040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:51.123780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f52b8-dd21-4ae6-bff2-0bb078738040 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:51.127387 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f1710-58b4-4ba8-8f66-3a72118ec05a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:51.128037 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f1710-58b4-4ba8-8f66-3a72118ec05a service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:51.128371 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f1710-58b4-4ba8-8f66-3a72118ec05a service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:51.128700 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f1710-58b4-4ba8-8f66-3a72118ec05a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:51.138510 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f1710-58b4-4ba8-8f66-3a72118ec05a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:51.139979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f1710-58b4-4ba8-8f66-3a72118ec05a service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:51.150035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1370/2778] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:16:51.780111 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c56f10-eeb8-4cb1-91c0-45b56774efe3 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:51.780436 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c56f10-eeb8-4cb1-91c0-45b56774efe3 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:51.780804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c56f10-eeb8-4cb1-91c0-45b56774efe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:51.809637 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-20c56f10-eeb8-4cb1-91c0-45b56774efe3 None None] MFA Rules not processed for user `25611f981ea74fe595c655a01f4aadcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:51.810848 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-20c56f10-eeb8-4cb1-91c0-45b56774efe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:51.834805 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:51.883676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1409/2779] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:51.888612 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f1710-58b4-4ba8-8f66-3a72118ec05a service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:51.902860 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f1710-58b4-4ba8-8f66-3a72118ec05a service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:51.908538 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd51f8-3772-41d5-bda8-20d9b8294289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:51.908877 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd51f8-3772-41d5-bda8-20d9b8294289 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:51.909855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd51f8-3772-41d5-bda8-20d9b8294289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:51.910220 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fd51f8-3772-41d5-bda8-20d9b8294289 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:51.910923 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd51f8-3772-41d5-bda8-20d9b8294289 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:51.912970 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fd51f8-3772-41d5-bda8-20d9b8294289 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:51.918834 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:51.948001 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1371/2780] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:16:51 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:51.952944 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c56f10-eeb8-4cb1-91c0-45b56774efe3 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:51.969175 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c56f10-eeb8-4cb1-91c0-45b56774efe3 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:52.036361 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d055c-4230-48a7-9de5-a82c8f1681b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:52.036986 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d055c-4230-48a7-9de5-a82c8f1681b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:52.037266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d055c-4230-48a7-9de5-a82c8f1681b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:52.037537 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d055c-4230-48a7-9de5-a82c8f1681b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:52.038156 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d055c-4230-48a7-9de5-a82c8f1681b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:52.040356 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d055c-4230-48a7-9de5-a82c8f1681b8 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:52.061421 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1410/2781] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:16:51 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:52.065870 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd51f8-3772-41d5-bda8-20d9b8294289 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:52.076688 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fd51f8-3772-41d5-bda8-20d9b8294289 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:52.082008 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2ad5e-8c45-46ba-bac6-1a79d5a4ae8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:52.082613 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2ad5e-8c45-46ba-bac6-1a79d5a4ae8f admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:52.082894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2ad5e-8c45-46ba-bac6-1a79d5a4ae8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:52.083509 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2ad5e-8c45-46ba-bac6-1a79d5a4ae8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:52.084165 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2ad5e-8c45-46ba-bac6-1a79d5a4ae8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:52.085780 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2ad5e-8c45-46ba-bac6-1a79d5a4ae8f admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:52.096156 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1372/2782] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:52.100618 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d055c-4230-48a7-9de5-a82c8f1681b8 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:52.110458 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d055c-4230-48a7-9de5-a82c8f1681b8 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:52.115058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc0933-4a18-4705-b907-8c5acbdc86da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:52.115723 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc0933-4a18-4705-b907-8c5acbdc86da admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:52.116001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc0933-4a18-4705-b907-8c5acbdc86da admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:52.116270 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bc0933-4a18-4705-b907-8c5acbdc86da admin admin] PATH_INFO: `/v3/projects/9135d495ea9f434a88f2011f1daeb1f9/users/220360d2985d4c7f853c0a874eb829b6/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:52.131681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc0933-4a18-4705-b907-8c5acbdc86da admin admin] RBAC: Authorizing `identity:create_grant(project_id=9135d495ea9f434a88f2011f1daeb1f9, user_id=220360d2985d4c7f853c0a874eb829b6, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:52.134334 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bc0933-4a18-4705-b907-8c5acbdc86da admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:52.147355 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1411/2783] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:52 2022] PUT /identity/v3/projects/9135d495ea9f434a88f2011f1daeb1f9/users/220360d2985d4c7f853c0a874eb829b6/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:52.153591 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2ad5e-8c45-46ba-bac6-1a79d5a4ae8f admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:52.163564 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2ad5e-8c45-46ba-bac6-1a79d5a4ae8f admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:52.221811 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b00192-cd71-4463-97df-441d1aeb17f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:52.222554 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b00192-cd71-4463-97df-441d1aeb17f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:52.222899 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b00192-cd71-4463-97df-441d1aeb17f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:52.223206 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b00192-cd71-4463-97df-441d1aeb17f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:52.223850 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b00192-cd71-4463-97df-441d1aeb17f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:52.225419 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b00192-cd71-4463-97df-441d1aeb17f4 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:52.236309 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1373/2784] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:16:52 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:16:52.242171 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc0933-4a18-4705-b907-8c5acbdc86da admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:52.260234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bc0933-4a18-4705-b907-8c5acbdc86da admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:52.265271 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d39a3-c777-445f-a099-9d5183520810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:52.266176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d39a3-c777-445f-a099-9d5183520810 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:52.266463 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d39a3-c777-445f-a099-9d5183520810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:52.266806 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d39a3-c777-445f-a099-9d5183520810 admin admin] PATH_INFO: `/v3/projects/9135d495ea9f434a88f2011f1daeb1f9/users/220360d2985d4c7f853c0a874eb829b6/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:52.269777 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d39a3-c777-445f-a099-9d5183520810 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9135d495ea9f434a88f2011f1daeb1f9, user_id=220360d2985d4c7f853c0a874eb829b6, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:52.272735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d39a3-c777-445f-a099-9d5183520810 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:52.283815 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1412/2785] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:16:52 2022] PUT /identity/v3/projects/9135d495ea9f434a88f2011f1daeb1f9/users/220360d2985d4c7f853c0a874eb829b6/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:16:52.921416 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7afa0-5202-4546-a440-d36a0ac853c1 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:52.921962 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7afa0-5202-4546-a440-d36a0ac853c1 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:52.921962 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7afa0-5202-4546-a440-d36a0ac853c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:52.950552 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-3bd7afa0-5202-4546-a440-d36a0ac853c1 None None] MFA Rules not processed for user `220360d2985d4c7f853c0a874eb829b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:52.951759 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-3bd7afa0-5202-4546-a440-d36a0ac853c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:53.025326 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1374/2786] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:16:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:16:53.037288 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d39a3-c777-445f-a099-9d5183520810 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:53.039185 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-355d39a3-c777-445f-a099-9d5183520810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:53.100032 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d39a3-c777-445f-a099-9d5183520810 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:53.103583 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5721934-137a-441e-b102-78328b45f99c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:53.104184 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5721934-137a-441e-b102-78328b45f99c service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:53.104459 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5721934-137a-441e-b102-78328b45f99c service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:53.104725 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5721934-137a-441e-b102-78328b45f99c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:53.132840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:53.161685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5721934-137a-441e-b102-78328b45f99c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:53.163242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5721934-137a-441e-b102-78328b45f99c service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:53.172270 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1413/2787] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:16:54.526225 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7afa0-5202-4546-a440-d36a0ac853c1 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:54.527706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-3bd7afa0-5202-4546-a440-d36a0ac853c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:54.540234 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7afa0-5202-4546-a440-d36a0ac853c1 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:54.543975 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112f668-1ff7-4d97-84cd-21138acd2f6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:54.544660 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112f668-1ff7-4d97-84cd-21138acd2f6d service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:54.544925 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112f668-1ff7-4d97-84cd-21138acd2f6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:54.545192 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112f668-1ff7-4d97-84cd-21138acd2f6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:54.554942 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112f668-1ff7-4d97-84cd-21138acd2f6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:54.556427 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112f668-1ff7-4d97-84cd-21138acd2f6d service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:54.566918 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1375/2788] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:16:54 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:17:03.167991 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=14 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:04.561585 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=79,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:10.321655 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5721934-137a-441e-b102-78328b45f99c service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:10.323056 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f5721934-137a-441e-b102-78328b45f99c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:10.388922 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5721934-137a-441e-b102-78328b45f99c service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:10.393396 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d3c0e-6064-44f4-ac9d-e6ccc25db706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:10.394023 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d3c0e-6064-44f4-ac9d-e6ccc25db706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:10.394306 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d3c0e-6064-44f4-ac9d-e6ccc25db706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:10.394597 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d3c0e-6064-44f4-ac9d-e6ccc25db706 admin admin] PATH_INFO: `/v3/users/6b397b9393814b88bd78b45eef386573` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:10.395753 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d3c0e-6064-44f4-ac9d-e6ccc25db706 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b397b9393814b88bd78b45eef386573)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:10.397653 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d3c0e-6064-44f4-ac9d-e6ccc25db706 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:10.442607 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-814d3c0e-6064-44f4-ac9d-e6ccc25db706 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b397b9393814b88bd78b45eef386573'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:10.449329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-814d3c0e-6064-44f4-ac9d-e6ccc25db706 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b397b9393814b88bd78b45eef386573'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:10.456193 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-814d3c0e-6064-44f4-ac9d-e6ccc25db706 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b397b9393814b88bd78b45eef386573'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:10.456796 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-814d3c0e-6064-44f4-ac9d-e6ccc25db706 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b397b9393814b88bd78b45eef386573'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:10.457388 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-814d3c0e-6064-44f4-ac9d-e6ccc25db706 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b397b9393814b88bd78b45eef386573'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:10.475093 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1414/2789] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:17:10 2022] DELETE /identity/v3/users/6b397b9393814b88bd78b45eef386573 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:10.479711 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f4f8a-9752-431e-8ebe-77194de1cf25 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:10.480231 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f4f8a-9752-431e-8ebe-77194de1cf25 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:10.480335 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f4f8a-9752-431e-8ebe-77194de1cf25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:10.499436 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-897f4f8a-9752-431e-8ebe-77194de1cf25 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:10.500630 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-897f4f8a-9752-431e-8ebe-77194de1cf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:10.557897 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1376/2790] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:10.567551 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d3c0e-6064-44f4-ac9d-e6ccc25db706 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:10.568953 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-814d3c0e-6064-44f4-ac9d-e6ccc25db706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:10.644075 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d3c0e-6064-44f4-ac9d-e6ccc25db706 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:10.647662 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91843d4b-23fd-4943-9019-af2d84780b2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:10.648248 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91843d4b-23fd-4943-9019-af2d84780b2b service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:10.648522 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91843d4b-23fd-4943-9019-af2d84780b2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:10.648788 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91843d4b-23fd-4943-9019-af2d84780b2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:10.657455 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91843d4b-23fd-4943-9019-af2d84780b2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:10.659390 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91843d4b-23fd-4943-9019-af2d84780b2b service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:10.668608 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1415/2791] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:17:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:17:10.951562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f4f8a-9752-431e-8ebe-77194de1cf25 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:10.953015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-897f4f8a-9752-431e-8ebe-77194de1cf25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:10.965445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f4f8a-9752-431e-8ebe-77194de1cf25 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:10.970587 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c49d56-4f2c-4d1a-9b60-0e3d4f292eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:10.971265 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c49d56-4f2c-4d1a-9b60-0e3d4f292eee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:10.971655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c49d56-4f2c-4d1a-9b60-0e3d4f292eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:10.972147 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c49d56-4f2c-4d1a-9b60-0e3d4f292eee admin admin] PATH_INFO: `/v3/projects/6f7eb94d4f644255a15f6afbe20b64b1` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:10.973989 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c49d56-4f2c-4d1a-9b60-0e3d4f292eee admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f7eb94d4f644255a15f6afbe20b64b1)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:10.975937 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c49d56-4f2c-4d1a-9b60-0e3d4f292eee admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:11.022573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-e2c49d56-4f2c-4d1a-9b60-0e3d4f292eee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f7eb94d4f644255a15f6afbe20b64b1', 'request_id': 'req-e2c49d56-4f2c-4d1a-9b60-0e3d4f292eee'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:11.029672 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-e2c49d56-4f2c-4d1a-9b60-0e3d4f292eee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f7eb94d4f644255a15f6afbe20b64b1', 'request_id': 'req-e2c49d56-4f2c-4d1a-9b60-0e3d4f292eee'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:11.035323 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-e2c49d56-4f2c-4d1a-9b60-0e3d4f292eee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f7eb94d4f644255a15f6afbe20b64b1', 'request_id': 'req-e2c49d56-4f2c-4d1a-9b60-0e3d4f292eee'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:11.047894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-e2c49d56-4f2c-4d1a-9b60-0e3d4f292eee admin admin] The token cache is being invalidate because project 6f7eb94d4f644255a15f6afbe20b64b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:17:11.048191 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-e2c49d56-4f2c-4d1a-9b60-0e3d4f292eee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:11.048828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-e2c49d56-4f2c-4d1a-9b60-0e3d4f292eee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:11.051251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1377/2792] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:17:10 2022] DELETE /identity/v3/projects/6f7eb94d4f644255a15f6afbe20b64b1 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:11.059549 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9890f-ec0a-43bb-9ff9-0ecec93933f4 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:11.060168 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9890f-ec0a-43bb-9ff9-0ecec93933f4 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:11.060495 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9890f-ec0a-43bb-9ff9-0ecec93933f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:11.069610 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:11.084041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-02a9890f-ec0a-43bb-9ff9-0ecec93933f4 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:11.085224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-02a9890f-ec0a-43bb-9ff9-0ecec93933f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:11.143340 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1416/2793] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:17:11.151174 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd91dc-c899-4997-98d1-3011575b9943 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:11.151463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd91dc-c899-4997-98d1-3011575b9943 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:11.151740 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd91dc-c899-4997-98d1-3011575b9943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:11.171628 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-54dd91dc-c899-4997-98d1-3011575b9943 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:11.172830 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-54dd91dc-c899-4997-98d1-3011575b9943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:11.181496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1378/2794] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:11.186910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9890f-ec0a-43bb-9ff9-0ecec93933f4 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:11.196321 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9890f-ec0a-43bb-9ff9-0ecec93933f4 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:11.200830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f27116-f6a5-47d2-b81a-94472a0caa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:11.201419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f27116-f6a5-47d2-b81a-94472a0caa8d admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:11.201695 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f27116-f6a5-47d2-b81a-94472a0caa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:11.201963 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f27116-f6a5-47d2-b81a-94472a0caa8d admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:11.202592 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f27116-f6a5-47d2-b81a-94472a0caa8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:11.202890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f27116-f6a5-47d2-b81a-94472a0caa8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:11.204497 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f27116-f6a5-47d2-b81a-94472a0caa8d admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:11.215879 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1417/2795] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:11.220147 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd91dc-c899-4997-98d1-3011575b9943 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:11.229795 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd91dc-c899-4997-98d1-3011575b9943 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:11.234527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673aa85-e41e-4c62-a344-e65b41f603bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:11.235147 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673aa85-e41e-4c62-a344-e65b41f603bb admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:11.235432 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673aa85-e41e-4c62-a344-e65b41f603bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:11.235708 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8673aa85-e41e-4c62-a344-e65b41f603bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:11.236344 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673aa85-e41e-4c62-a344-e65b41f603bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:11.238817 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8673aa85-e41e-4c62-a344-e65b41f603bb admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:11.264842 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1379/2796] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:17:11 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:17:11.268997 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f27116-f6a5-47d2-b81a-94472a0caa8d admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:11.278844 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f27116-f6a5-47d2-b81a-94472a0caa8d admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:11.332322 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86fb7c-4fd0-4727-af4f-4f8f6087d1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:11.332926 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86fb7c-4fd0-4727-af4f-4f8f6087d1f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:11.333109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86fb7c-4fd0-4727-af4f-4f8f6087d1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:11.333376 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e86fb7c-4fd0-4727-af4f-4f8f6087d1f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:11.334253 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e86fb7c-4fd0-4727-af4f-4f8f6087d1f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:11.336176 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e86fb7c-4fd0-4727-af4f-4f8f6087d1f2 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:11.358001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1418/2797] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:17:11 2022] POST /identity/v3/users => generated 288 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:17:11.361549 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673aa85-e41e-4c62-a344-e65b41f603bb admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:11.372359 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8673aa85-e41e-4c62-a344-e65b41f603bb admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:11.377074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884681b-0c0a-4e60-a3b7-1976fa5b305c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:11.377720 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884681b-0c0a-4e60-a3b7-1976fa5b305c admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:11.378001 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884681b-0c0a-4e60-a3b7-1976fa5b305c admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:11.378274 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884681b-0c0a-4e60-a3b7-1976fa5b305c admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:11.378911 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884681b-0c0a-4e60-a3b7-1976fa5b305c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:11.380530 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884681b-0c0a-4e60-a3b7-1976fa5b305c admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:11.390200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1380/2798] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:17:11.394297 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86fb7c-4fd0-4727-af4f-4f8f6087d1f2 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:11.404030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e86fb7c-4fd0-4727-af4f-4f8f6087d1f2 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:11.408712 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265deef6-d383-484d-bf9f-1c8a3bb45512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:11.409808 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265deef6-d383-484d-bf9f-1c8a3bb45512 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:11.410095 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265deef6-d383-484d-bf9f-1c8a3bb45512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:11.410393 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265deef6-d383-484d-bf9f-1c8a3bb45512 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:11.411029 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265deef6-d383-484d-bf9f-1c8a3bb45512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:11.412637 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265deef6-d383-484d-bf9f-1c8a3bb45512 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:11.423114 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1419/2799] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:17:11.427701 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884681b-0c0a-4e60-a3b7-1976fa5b305c admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:11.437565 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884681b-0c0a-4e60-a3b7-1976fa5b305c admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:11.442246 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef26c5-7868-4420-a3db-ee78b28274ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:11.442884 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef26c5-7868-4420-a3db-ee78b28274ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:11.443193 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef26c5-7868-4420-a3db-ee78b28274ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:11.443463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ef26c5-7868-4420-a3db-ee78b28274ab admin admin] PATH_INFO: `/v3/projects/a09b3fcd4fdc424086b7ca6b15d0a4e9/users/7e1f6e80465a44d0a90200ada3f43e23/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:11.458111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef26c5-7868-4420-a3db-ee78b28274ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=a09b3fcd4fdc424086b7ca6b15d0a4e9, user_id=7e1f6e80465a44d0a90200ada3f43e23, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:11.461007 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ef26c5-7868-4420-a3db-ee78b28274ab admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:11.474021 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1381/2800] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:17:11 2022] PUT /identity/v3/projects/a09b3fcd4fdc424086b7ca6b15d0a4e9/users/7e1f6e80465a44d0a90200ada3f43e23/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:11.478993 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043739e-a610-4241-be14-980f71e86029 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:11.479309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043739e-a610-4241-be14-980f71e86029 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:11.479580 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043739e-a610-4241-be14-980f71e86029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:11.498831 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-0043739e-a610-4241-be14-980f71e86029 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:11.500033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0043739e-a610-4241-be14-980f71e86029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:11.556847 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1420/2801] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:11.566512 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef26c5-7868-4420-a3db-ee78b28274ab admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:11.568312 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-22ef26c5-7868-4420-a3db-ee78b28274ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:11.590179 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:11.625847 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ef26c5-7868-4420-a3db-ee78b28274ab admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:11.629521 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041c1a8-5a96-43ad-b9af-8739add0c221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:11.630116 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041c1a8-5a96-43ad-b9af-8739add0c221 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:11.630393 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041c1a8-5a96-43ad-b9af-8739add0c221 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:11.630661 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5041c1a8-5a96-43ad-b9af-8739add0c221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:11.639368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041c1a8-5a96-43ad-b9af-8739add0c221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:11.640764 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5041c1a8-5a96-43ad-b9af-8739add0c221 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:11.650459 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1382/2802] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:17:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:17:15.953011 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591ac47-4520-4c5e-8b5f-9c1fd797c5f8 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:15.953558 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591ac47-4520-4c5e-8b5f-9c1fd797c5f8 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:15.953946 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591ac47-4520-4c5e-8b5f-9c1fd797c5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:15.991376 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:16.004344 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-b591ac47-4520-4c5e-8b5f-9c1fd797c5f8 None None] MFA Rules not processed for user `7e1f6e80465a44d0a90200ada3f43e23`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:16.005829 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-b591ac47-4520-4c5e-8b5f-9c1fd797c5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:16.081901 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1421/2803] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:16.086948 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041c1a8-5a96-43ad-b9af-8739add0c221 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:16.097620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5041c1a8-5a96-43ad-b9af-8739add0c221 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:16.102812 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40203dcd-69cb-4d19-a36d-944cef1b8e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:16.103603 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40203dcd-69cb-4d19-a36d-944cef1b8e0a admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:16.103909 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40203dcd-69cb-4d19-a36d-944cef1b8e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:16.104457 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40203dcd-69cb-4d19-a36d-944cef1b8e0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:16.106392 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40203dcd-69cb-4d19-a36d-944cef1b8e0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:16.108361 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40203dcd-69cb-4d19-a36d-944cef1b8e0a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:16.148620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1383/2804] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:17:16 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:17:16.152730 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591ac47-4520-4c5e-8b5f-9c1fd797c5f8 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:16.164522 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591ac47-4520-4c5e-8b5f-9c1fd797c5f8 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:16.230948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff01d0b-2eea-4a6b-bd6e-abb6acc73704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:16.231654 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff01d0b-2eea-4a6b-bd6e-abb6acc73704 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:16.231960 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff01d0b-2eea-4a6b-bd6e-abb6acc73704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:16.232262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff01d0b-2eea-4a6b-bd6e-abb6acc73704 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:16.232962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff01d0b-2eea-4a6b-bd6e-abb6acc73704 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:16.235031 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff01d0b-2eea-4a6b-bd6e-abb6acc73704 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:16.263763 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1422/2805] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:17:16 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:17:16.267530 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40203dcd-69cb-4d19-a36d-944cef1b8e0a admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:16.280074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40203dcd-69cb-4d19-a36d-944cef1b8e0a admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:16.285509 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cca6cc-c444-47c1-a544-ce3f7e43805f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:16.286621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cca6cc-c444-47c1-a544-ce3f7e43805f admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:16.287259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cca6cc-c444-47c1-a544-ce3f7e43805f admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:16.287739 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cca6cc-c444-47c1-a544-ce3f7e43805f admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:16.288452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cca6cc-c444-47c1-a544-ce3f7e43805f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:16.290245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cca6cc-c444-47c1-a544-ce3f7e43805f admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:16.303580 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1384/2806] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:17:16.308149 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff01d0b-2eea-4a6b-bd6e-abb6acc73704 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:16.326350 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff01d0b-2eea-4a6b-bd6e-abb6acc73704 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:16.331564 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8135cb9-0240-4bc5-97d9-6b08b9589a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:16.332427 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8135cb9-0240-4bc5-97d9-6b08b9589a00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:16.332726 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8135cb9-0240-4bc5-97d9-6b08b9589a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:16.333005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8135cb9-0240-4bc5-97d9-6b08b9589a00 admin admin] PATH_INFO: `/v3/projects/7d739cfc2558439ea68522c1ecc46287/users/2b62d2360a2a4e278b45d1a5c45e0cd9/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:16.365018 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8135cb9-0240-4bc5-97d9-6b08b9589a00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d739cfc2558439ea68522c1ecc46287, user_id=2b62d2360a2a4e278b45d1a5c45e0cd9, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:16.367817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8135cb9-0240-4bc5-97d9-6b08b9589a00 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:16.382490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1423/2807] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:17:16 2022] PUT /identity/v3/projects/7d739cfc2558439ea68522c1ecc46287/users/2b62d2360a2a4e278b45d1a5c45e0cd9/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:16.386285 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cca6cc-c444-47c1-a544-ce3f7e43805f admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:16.396040 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cca6cc-c444-47c1-a544-ce3f7e43805f admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:16.453111 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ef681-e201-43fa-839f-ac0011a28ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:16.453715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ef681-e201-43fa-839f-ac0011a28ffe admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:16.453995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ef681-e201-43fa-839f-ac0011a28ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:16.454267 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ef681-e201-43fa-839f-ac0011a28ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:16.454904 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ef681-e201-43fa-839f-ac0011a28ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:16.456607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ef681-e201-43fa-839f-ac0011a28ffe admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:16.466393 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1385/2808] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:17:16.470483 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8135cb9-0240-4bc5-97d9-6b08b9589a00 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:16.481044 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8135cb9-0240-4bc5-97d9-6b08b9589a00 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:16.485770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225b405-dc3c-4f40-9db1-ed315e0d0e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:16.486442 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225b405-dc3c-4f40-9db1-ed315e0d0e99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:16.486718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225b405-dc3c-4f40-9db1-ed315e0d0e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:16.487595 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225b405-dc3c-4f40-9db1-ed315e0d0e99 admin admin] PATH_INFO: `/v3/projects/7d739cfc2558439ea68522c1ecc46287/users/2b62d2360a2a4e278b45d1a5c45e0cd9/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:16.489585 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225b405-dc3c-4f40-9db1-ed315e0d0e99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d739cfc2558439ea68522c1ecc46287, user_id=2b62d2360a2a4e278b45d1a5c45e0cd9, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:16.492296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4225b405-dc3c-4f40-9db1-ed315e0d0e99 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:16.502910 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1424/2809] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:17:16 2022] PUT /identity/v3/projects/7d739cfc2558439ea68522c1ecc46287/users/2b62d2360a2a4e278b45d1a5c45e0cd9/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:18.808944 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ef681-e201-43fa-839f-ac0011a28ffe admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:18.818349 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ad6ef681-e201-43fa-839f-ac0011a28ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:18.884113 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225b405-dc3c-4f40-9db1-ed315e0d0e99 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:18.885875 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-4225b405-dc3c-4f40-9db1-ed315e0d0e99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:19.013319 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:19.044677 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225b405-dc3c-4f40-9db1-ed315e0d0e99 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:19.050681 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f828e4-fc03-42b8-93cd-d20a0ef4952a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:19.051492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f828e4-fc03-42b8-93cd-d20a0ef4952a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:19.052945 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f828e4-fc03-42b8-93cd-d20a0ef4952a admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:19.053228 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f828e4-fc03-42b8-93cd-d20a0ef4952a admin admin] PATH_INFO: `/v3/users/25611f981ea74fe595c655a01f4aadcc` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:19.054726 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f828e4-fc03-42b8-93cd-d20a0ef4952a admin admin] RBAC: Authorizing `identity:delete_user(user_id=25611f981ea74fe595c655a01f4aadcc)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:19.056771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f828e4-fc03-42b8-93cd-d20a0ef4952a admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:19.185930 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-93f828e4-fc03-42b8-93cd-d20a0ef4952a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25611f981ea74fe595c655a01f4aadcc'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:19.203740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-93f828e4-fc03-42b8-93cd-d20a0ef4952a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25611f981ea74fe595c655a01f4aadcc'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:19.214178 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-93f828e4-fc03-42b8-93cd-d20a0ef4952a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25611f981ea74fe595c655a01f4aadcc'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:19.215315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-93f828e4-fc03-42b8-93cd-d20a0ef4952a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25611f981ea74fe595c655a01f4aadcc'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:19.217234 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-93f828e4-fc03-42b8-93cd-d20a0ef4952a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25611f981ea74fe595c655a01f4aadcc'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:19.258640 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1425/2810] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:17:18 2022] DELETE /identity/v3/users/25611f981ea74fe595c655a01f4aadcc => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:17:19.260830 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f828e4-fc03-42b8-93cd-d20a0ef4952a admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:19.274766 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-93f828e4-fc03-42b8-93cd-d20a0ef4952a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:19.317420 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ef681-e201-43fa-839f-ac0011a28ffe admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:19.335320 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:19.435197 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f828e4-fc03-42b8-93cd-d20a0ef4952a admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:19.450182 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4bac1-1c50-4dee-b6bb-26555235549a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:19.450931 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4bac1-1c50-4dee-b6bb-26555235549a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:19.451255 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4bac1-1c50-4dee-b6bb-26555235549a admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:19.451539 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4bac1-1c50-4dee-b6bb-26555235549a admin admin] PATH_INFO: `/v3/users/a39a21bdfd944434a8b60fa90f9553e4` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:19.457712 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4bac1-1c50-4dee-b6bb-26555235549a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a39a21bdfd944434a8b60fa90f9553e4)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:19.464336 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db4bac1-1c50-4dee-b6bb-26555235549a admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:19.480282 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f35da9-c207-4efb-90ec-3ba4da929b75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a0f0fdbbd20492d8c35ec4b73404632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:19.481390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f35da9-c207-4efb-90ec-3ba4da929b75 service placement] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:19.481390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f35da9-c207-4efb-90ec-3ba4da929b75 service placement] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:19.481599 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f35da9-c207-4efb-90ec-3ba4da929b75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:19.483280 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-84f35da9-c207-4efb-90ec-3ba4da929b75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:19.509294 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f35da9-c207-4efb-90ec-3ba4da929b75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:19.510898 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f35da9-c207-4efb-90ec-3ba4da929b75 service placement] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:19.527987 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1386/2811] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:17:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 720 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:17:19.548950 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5db4bac1-1c50-4dee-b6bb-26555235549a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a39a21bdfd944434a8b60fa90f9553e4'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:19.568924 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5db4bac1-1c50-4dee-b6bb-26555235549a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a39a21bdfd944434a8b60fa90f9553e4'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:19.589940 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5db4bac1-1c50-4dee-b6bb-26555235549a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a39a21bdfd944434a8b60fa90f9553e4'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:19.593568 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5db4bac1-1c50-4dee-b6bb-26555235549a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a39a21bdfd944434a8b60fa90f9553e4'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:19.594515 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5db4bac1-1c50-4dee-b6bb-26555235549a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a39a21bdfd944434a8b60fa90f9553e4'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:19.604569 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:19.619430 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1426/2812] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:17:19 2022] DELETE /identity/v3/users/a39a21bdfd944434a8b60fa90f9553e4 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:19.624172 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8816e-fec9-4c1e-8df0-c4a9d04bc101 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:19.624759 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8816e-fec9-4c1e-8df0-c4a9d04bc101 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:19.625185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8816e-fec9-4c1e-8df0-c4a9d04bc101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:19.682300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4bac1-1c50-4dee-b6bb-26555235549a admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:19.686770 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5db4bac1-1c50-4dee-b6bb-26555235549a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:19.729285 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-a3e8816e-fec9-4c1e-8df0-c4a9d04bc101 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:19.730718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-a3e8816e-fec9-4c1e-8df0-c4a9d04bc101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:19.858146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4bac1-1c50-4dee-b6bb-26555235549a admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:19.868324 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60377685-db26-40c3-9021-87f86c0ee84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:19.868992 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60377685-db26-40c3-9021-87f86c0ee84c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:19.869274 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60377685-db26-40c3-9021-87f86c0ee84c admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:19.869546 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60377685-db26-40c3-9021-87f86c0ee84c admin admin] PATH_INFO: `/v3/projects/09cf446b5e344735846f09b4de104575` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:19.871401 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60377685-db26-40c3-9021-87f86c0ee84c admin admin] RBAC: Authorizing `identity:delete_project(project_id=09cf446b5e344735846f09b4de104575)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:19.873986 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60377685-db26-40c3-9021-87f86c0ee84c admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:19.885935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1387/2813] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:19.896983 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8816e-fec9-4c1e-8df0-c4a9d04bc101 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:19.898546 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-a3e8816e-fec9-4c1e-8df0-c4a9d04bc101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:19.987823 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-60377685-db26-40c3-9021-87f86c0ee84c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09cf446b5e344735846f09b4de104575', 'request_id': 'req-60377685-db26-40c3-9021-87f86c0ee84c'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:20.000577 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-60377685-db26-40c3-9021-87f86c0ee84c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09cf446b5e344735846f09b4de104575', 'request_id': 'req-60377685-db26-40c3-9021-87f86c0ee84c'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:20.008162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-60377685-db26-40c3-9021-87f86c0ee84c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09cf446b5e344735846f09b4de104575', 'request_id': 'req-60377685-db26-40c3-9021-87f86c0ee84c'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:20.023433 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-60377685-db26-40c3-9021-87f86c0ee84c admin admin] The token cache is being invalidate because project 09cf446b5e344735846f09b4de104575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:17:20.024007 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-60377685-db26-40c3-9021-87f86c0ee84c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:20.024928 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-60377685-db26-40c3-9021-87f86c0ee84c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:20.027246 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1427/2814] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:17:19 2022] DELETE /identity/v3/projects/09cf446b5e344735846f09b4de104575 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:20.032603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60377685-db26-40c3-9021-87f86c0ee84c admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:20.034848 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-60377685-db26-40c3-9021-87f86c0ee84c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:20.048579 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e8816e-fec9-4c1e-8df0-c4a9d04bc101 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:20.159761 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e817089-1d4e-4028-9dae-3c643d297fa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:20.160436 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e817089-1d4e-4028-9dae-3c643d297fa4 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:20.160628 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e817089-1d4e-4028-9dae-3c643d297fa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:20.160959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e817089-1d4e-4028-9dae-3c643d297fa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:20.165018 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-4e817089-1d4e-4028-9dae-3c643d297fa4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:20.191383 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60377685-db26-40c3-9021-87f86c0ee84c admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:20.191649 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e817089-1d4e-4028-9dae-3c643d297fa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:20.195500 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e817089-1d4e-4028-9dae-3c643d297fa4 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:20.202648 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91adb1-5e04-4386-aa1f-3f0afe632444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:20.205600 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91adb1-5e04-4386-aa1f-3f0afe632444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:20.206468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:20.207170 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91adb1-5e04-4386-aa1f-3f0afe632444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:20.208141 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c91adb1-5e04-4386-aa1f-3f0afe632444 admin admin] PATH_INFO: `/v3/users/220360d2985d4c7f853c0a874eb829b6` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:20.212518 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c91adb1-5e04-4386-aa1f-3f0afe632444 admin admin] RBAC: Authorizing `identity:delete_user(user_id=220360d2985d4c7f853c0a874eb829b6)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:20.220688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c91adb1-5e04-4386-aa1f-3f0afe632444 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:20.239292 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1388/2815] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:17:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:17:20.294341 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:20.363124 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-8c91adb1-5e04-4386-aa1f-3f0afe632444 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '220360d2985d4c7f853c0a874eb829b6'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:20.383695 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-8c91adb1-5e04-4386-aa1f-3f0afe632444 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '220360d2985d4c7f853c0a874eb829b6'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:20.407543 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-8c91adb1-5e04-4386-aa1f-3f0afe632444 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '220360d2985d4c7f853c0a874eb829b6'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:20.408766 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-8c91adb1-5e04-4386-aa1f-3f0afe632444 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '220360d2985d4c7f853c0a874eb829b6'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:20.409478 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-8c91adb1-5e04-4386-aa1f-3f0afe632444 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '220360d2985d4c7f853c0a874eb829b6'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:20.458736 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1428/2816] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:17:20 2022] DELETE /identity/v3/users/220360d2985d4c7f853c0a874eb829b6 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:20.854768 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e817089-1d4e-4028-9dae-3c643d297fa4 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:20.875008 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e817089-1d4e-4028-9dae-3c643d297fa4 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:20.916185 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91adb1-5e04-4386-aa1f-3f0afe632444 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:20.917939 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8c91adb1-5e04-4386-aa1f-3f0afe632444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:21.068671 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05c9c7-0a0b-4faf-9639-362ca2f15b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:21.070245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05c9c7-0a0b-4faf-9639-362ca2f15b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:21.070539 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05c9c7-0a0b-4faf-9639-362ca2f15b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:21.070814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05c9c7-0a0b-4faf-9639-362ca2f15b81 admin admin] PATH_INFO: `/v3/projects/9135d495ea9f434a88f2011f1daeb1f9` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:21.072971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05c9c7-0a0b-4faf-9639-362ca2f15b81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9135d495ea9f434a88f2011f1daeb1f9)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:21.075193 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05c9c7-0a0b-4faf-9639-362ca2f15b81 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:21.093162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c91adb1-5e04-4386-aa1f-3f0afe632444 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:21.108634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c890fc40-de7f-41d5-8f10-c2659c6be458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:21.109968 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890fc40-de7f-41d5-8f10-c2659c6be458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:21.110477 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890fc40-de7f-41d5-8f10-c2659c6be458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:21.111611 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890fc40-de7f-41d5-8f10-c2659c6be458 admin admin] PATH_INFO: `/v3/projects/5e1246566ce046ef933251215b242a45` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:21.113215 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c890fc40-de7f-41d5-8f10-c2659c6be458 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e1246566ce046ef933251215b242a45)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:21.119161 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c890fc40-de7f-41d5-8f10-c2659c6be458 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:21.164428 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-6e05c9c7-0a0b-4faf-9639-362ca2f15b81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9135d495ea9f434a88f2011f1daeb1f9', 'request_id': 'req-6e05c9c7-0a0b-4faf-9639-362ca2f15b81'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:21.180607 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-6e05c9c7-0a0b-4faf-9639-362ca2f15b81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9135d495ea9f434a88f2011f1daeb1f9', 'request_id': 'req-6e05c9c7-0a0b-4faf-9639-362ca2f15b81'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:21.190638 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-6e05c9c7-0a0b-4faf-9639-362ca2f15b81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9135d495ea9f434a88f2011f1daeb1f9', 'request_id': 'req-6e05c9c7-0a0b-4faf-9639-362ca2f15b81'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:21.210898 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-6e05c9c7-0a0b-4faf-9639-362ca2f15b81 admin admin] The token cache is being invalidate because project 9135d495ea9f434a88f2011f1daeb1f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:17:21.211524 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-6e05c9c7-0a0b-4faf-9639-362ca2f15b81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:21.212677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-6e05c9c7-0a0b-4faf-9639-362ca2f15b81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:21.214457 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1389/2817] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:17:20 2022] DELETE /identity/v3/projects/9135d495ea9f434a88f2011f1daeb1f9 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:21.219285 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04297791-a7c2-4523-b4be-4496075fa778 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:21.219939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04297791-a7c2-4523-b4be-4496075fa778 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:21.221108 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04297791-a7c2-4523-b4be-4496075fa778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:21.233146 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-c890fc40-de7f-41d5-8f10-c2659c6be458 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e1246566ce046ef933251215b242a45', 'request_id': 'req-c890fc40-de7f-41d5-8f10-c2659c6be458'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:21.245224 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-c890fc40-de7f-41d5-8f10-c2659c6be458 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e1246566ce046ef933251215b242a45', 'request_id': 'req-c890fc40-de7f-41d5-8f10-c2659c6be458'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:21.262369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-c890fc40-de7f-41d5-8f10-c2659c6be458 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e1246566ce046ef933251215b242a45', 'request_id': 'req-c890fc40-de7f-41d5-8f10-c2659c6be458'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:21.269372 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-04297791-a7c2-4523-b4be-4496075fa778 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:21.270261 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-04297791-a7c2-4523-b4be-4496075fa778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:21.284470 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-c890fc40-de7f-41d5-8f10-c2659c6be458 admin admin] The token cache is being invalidate because project 5e1246566ce046ef933251215b242a45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:17:21.284755 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-c890fc40-de7f-41d5-8f10-c2659c6be458 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:21.286970 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-c890fc40-de7f-41d5-8f10-c2659c6be458 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:21.295558 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1429/2818] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:17:20 2022] DELETE /identity/v3/projects/5e1246566ce046ef933251215b242a45 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:21.297148 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba676de4-cf9a-4c7c-bbb6-6df9dd94a44c None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:21.298659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba676de4-cf9a-4c7c-bbb6-6df9dd94a44c None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:21.298950 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba676de4-cf9a-4c7c-bbb6-6df9dd94a44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:21.354294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-ba676de4-cf9a-4c7c-bbb6-6df9dd94a44c None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:21.355382 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-ba676de4-cf9a-4c7c-bbb6-6df9dd94a44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:21.435537 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:21.447137 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1390/2819] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:17:21.449898 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79398535-8984-47f8-9676-5b44155369a5 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:21.455046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79398535-8984-47f8-9676-5b44155369a5 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:21.457001 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79398535-8984-47f8-9676-5b44155369a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:21.511239 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-79398535-8984-47f8-9676-5b44155369a5 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:21.512503 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-79398535-8984-47f8-9676-5b44155369a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:21.539883 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1430/2820] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:21.540237 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1391/2821] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:21.561109 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba676de4-cf9a-4c7c-bbb6-6df9dd94a44c None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:21.597686 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba676de4-cf9a-4c7c-bbb6-6df9dd94a44c None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:21.614704 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58c114-c8b9-4263-ab27-53bc3f271305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:21.615567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58c114-c8b9-4263-ab27-53bc3f271305 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:21.616024 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58c114-c8b9-4263-ab27-53bc3f271305 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:21.616364 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa58c114-c8b9-4263-ab27-53bc3f271305 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:21.617175 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58c114-c8b9-4263-ab27-53bc3f271305 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:21.617602 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58c114-c8b9-4263-ab27-53bc3f271305 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:21.619443 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa58c114-c8b9-4263-ab27-53bc3f271305 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:21.621128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc7e77-d23a-46b8-92db-47e17e13e370 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:21.621416 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc7e77-d23a-46b8-92db-47e17e13e370 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:21.621692 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc7e77-d23a-46b8-92db-47e17e13e370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:21.646574 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-43cc7e77-d23a-46b8-92db-47e17e13e370 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:21.647933 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-43cc7e77-d23a-46b8-92db-47e17e13e370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:21.658291 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1431/2822] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:21.661541 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58c114-c8b9-4263-ab27-53bc3f271305 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:21.665555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1392/2823] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:17:21.671792 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc7e77-d23a-46b8-92db-47e17e13e370 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:21.676798 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa58c114-c8b9-4263-ab27-53bc3f271305 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:21.689005 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981f557-2625-4dfa-9fe8-ba7b30a2cd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:21.689696 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981f557-2625-4dfa-9fe8-ba7b30a2cd3b admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:21.690543 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981f557-2625-4dfa-9fe8-ba7b30a2cd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:21.690889 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981f557-2625-4dfa-9fe8-ba7b30a2cd3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:21.691755 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981f557-2625-4dfa-9fe8-ba7b30a2cd3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:21.694962 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc7e77-d23a-46b8-92db-47e17e13e370 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:21.697718 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981f557-2625-4dfa-9fe8-ba7b30a2cd3b admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:21.710925 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d54c7e-d437-4d04-a727-be80cc7641c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:21.712026 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d54c7e-d437-4d04-a727-be80cc7641c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:21.712393 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d54c7e-d437-4d04-a727-be80cc7641c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:21.712737 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d54c7e-d437-4d04-a727-be80cc7641c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:21.713632 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d54c7e-d437-4d04-a727-be80cc7641c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:21.714044 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d54c7e-d437-4d04-a727-be80cc7641c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:21.715903 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d54c7e-d437-4d04-a727-be80cc7641c7 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:21.741379 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1393/2825] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:17:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:21.743162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1432/2824] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:17:21 2022] POST /identity/v3/projects => generated 375 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:17:21.744316 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d54c7e-d437-4d04-a727-be80cc7641c7 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:21.745371 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981f557-2625-4dfa-9fe8-ba7b30a2cd3b admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:21.750028 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:21.762468 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d54c7e-d437-4d04-a727-be80cc7641c7 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:21.868242 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981f557-2625-4dfa-9fe8-ba7b30a2cd3b admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:21.876667 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398078bc-797b-40fe-ad0f-6f20475bbd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:21.877290 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398078bc-797b-40fe-ad0f-6f20475bbd5e admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:21.877612 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398078bc-797b-40fe-ad0f-6f20475bbd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:21.877890 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398078bc-797b-40fe-ad0f-6f20475bbd5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:21.878535 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398078bc-797b-40fe-ad0f-6f20475bbd5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:21.880492 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398078bc-797b-40fe-ad0f-6f20475bbd5e admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:21.883265 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3f3d4-e22c-4071-8c6a-faaf12c7d2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:21.884252 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3f3d4-e22c-4071-8c6a-faaf12c7d2e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:21.884705 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3f3d4-e22c-4071-8c6a-faaf12c7d2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:21.885134 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b3f3d4-e22c-4071-8c6a-faaf12c7d2e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:21.885959 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b3f3d4-e22c-4071-8c6a-faaf12c7d2e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:21.889463 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b3f3d4-e22c-4071-8c6a-faaf12c7d2e4 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:21.946000 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1394/2826] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:17:21 2022] POST /identity/v3/users => generated 288 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:17:21.946174 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1433/2827] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:17:21 2022] POST /identity/v3/projects => generated 399 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:17:21.946291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3f3d4-e22c-4071-8c6a-faaf12c7d2e4 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:21.947103 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398078bc-797b-40fe-ad0f-6f20475bbd5e admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:21.955536 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b3f3d4-e22c-4071-8c6a-faaf12c7d2e4 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:22.030041 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398078bc-797b-40fe-ad0f-6f20475bbd5e admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:22.036973 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cdfa8c-a120-4cef-85a9-20fc72f3883a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:22.038409 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdfa8c-a120-4cef-85a9-20fc72f3883a admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:22.038858 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdfa8c-a120-4cef-85a9-20fc72f3883a admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:22.039323 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdfa8c-a120-4cef-85a9-20fc72f3883a admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:22.040138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cdfa8c-a120-4cef-85a9-20fc72f3883a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:22.042735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cdfa8c-a120-4cef-85a9-20fc72f3883a admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:22.048570 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a3e0d-363b-42a6-aa51-13864b80d89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:22.049764 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a3e0d-363b-42a6-aa51-13864b80d89c admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:22.050058 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a3e0d-363b-42a6-aa51-13864b80d89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:22.050330 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a3e0d-363b-42a6-aa51-13864b80d89c admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:22.051442 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a3e0d-363b-42a6-aa51-13864b80d89c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:22.053388 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a3e0d-363b-42a6-aa51-13864b80d89c admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:22.064379 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1434/2828] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:17:21 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:17:22.066080 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cdfa8c-a120-4cef-85a9-20fc72f3883a admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:22.079287 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cdfa8c-a120-4cef-85a9-20fc72f3883a admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:22.088479 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363eabe0-3e03-42ce-b2fd-7696cc638683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:22.089403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363eabe0-3e03-42ce-b2fd-7696cc638683 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:22.089681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1395/2829] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:17:21 2022] POST /identity/v3/users => generated 300 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:17:22.090413 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363eabe0-3e03-42ce-b2fd-7696cc638683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:22.090833 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363eabe0-3e03-42ce-b2fd-7696cc638683 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:22.091909 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363eabe0-3e03-42ce-b2fd-7696cc638683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:22.093542 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a3e0d-363b-42a6-aa51-13864b80d89c admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:22.093842 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363eabe0-3e03-42ce-b2fd-7696cc638683 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:22.104651 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a3e0d-363b-42a6-aa51-13864b80d89c admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:22.108026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1435/2830] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:17:22.110460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597f820-afaf-4c4d-a1ad-eb8f0d00b527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:22.111265 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597f820-afaf-4c4d-a1ad-eb8f0d00b527 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:22.111405 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597f820-afaf-4c4d-a1ad-eb8f0d00b527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:22.111665 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9597f820-afaf-4c4d-a1ad-eb8f0d00b527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:22.112348 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9597f820-afaf-4c4d-a1ad-eb8f0d00b527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:22.112477 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363eabe0-3e03-42ce-b2fd-7696cc638683 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:22.113935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9597f820-afaf-4c4d-a1ad-eb8f0d00b527 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:22.123554 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:22.127788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1396/2831] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:17:22.128753 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363eabe0-3e03-42ce-b2fd-7696cc638683 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:22.133442 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597f820-afaf-4c4d-a1ad-eb8f0d00b527 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:22.135020 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40417cd4-e03a-4e54-968b-b86996cc6c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:22.135944 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40417cd4-e03a-4e54-968b-b86996cc6c8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:22.136298 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40417cd4-e03a-4e54-968b-b86996cc6c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:22.136641 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40417cd4-e03a-4e54-968b-b86996cc6c8e admin admin] PATH_INFO: `/v3/projects/83d70cb638b84a47bef3eec9df2bf372/users/1182c1072e5b4770a2c7eb303a7e3a4e/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:22.150251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9597f820-afaf-4c4d-a1ad-eb8f0d00b527 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:22.156909 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40417cd4-e03a-4e54-968b-b86996cc6c8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=83d70cb638b84a47bef3eec9df2bf372, user_id=1182c1072e5b4770a2c7eb303a7e3a4e, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:22.157097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740aa34-0ca0-4a88-a050-59cfd636924b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:22.157279 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740aa34-0ca0-4a88-a050-59cfd636924b admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:22.157557 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740aa34-0ca0-4a88-a050-59cfd636924b admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:22.158068 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a740aa34-0ca0-4a88-a050-59cfd636924b admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:22.159578 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740aa34-0ca0-4a88-a050-59cfd636924b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:22.160435 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40417cd4-e03a-4e54-968b-b86996cc6c8e admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:22.161158 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a740aa34-0ca0-4a88-a050-59cfd636924b admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:22.182118 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1397/2832] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:17:22 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:17:22.182644 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1436/2833] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:17:22 2022] PUT /identity/v3/projects/83d70cb638b84a47bef3eec9df2bf372/users/1182c1072e5b4770a2c7eb303a7e3a4e/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:17:22.188443 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740aa34-0ca0-4a88-a050-59cfd636924b admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:22.193590 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feed35e-5c6c-453e-bccf-afd90db97883 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:22.193982 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feed35e-5c6c-453e-bccf-afd90db97883 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:22.194359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feed35e-5c6c-453e-bccf-afd90db97883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:22.212425 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a740aa34-0ca0-4a88-a050-59cfd636924b admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:22.252136 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-4feed35e-5c6c-453e-bccf-afd90db97883 None None] MFA Rules not processed for user `1182c1072e5b4770a2c7eb303a7e3a4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:22.253888 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-4feed35e-5c6c-453e-bccf-afd90db97883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:22.318304 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ced6d-18fc-4953-a0f7-c2e1b601d351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:22.319000 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ced6d-18fc-4953-a0f7-c2e1b601d351 admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:22.319301 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ced6d-18fc-4953-a0f7-c2e1b601d351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:22.319576 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ced6d-18fc-4953-a0f7-c2e1b601d351 admin admin] PATH_INFO: `/v3/projects/eafb7937222b4225b2bfbb30ddfc1950/users/41244285613d4d2ea744668b8ae8e32c/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:22.322221 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:22.340858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ced6d-18fc-4953-a0f7-c2e1b601d351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eafb7937222b4225b2bfbb30ddfc1950, user_id=41244285613d4d2ea744668b8ae8e32c, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:22.348471 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ced6d-18fc-4953-a0f7-c2e1b601d351 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:22.354602 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1437/2834] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:22.368033 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feed35e-5c6c-453e-bccf-afd90db97883 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:22.370158 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-4feed35e-5c6c-453e-bccf-afd90db97883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:22.381585 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1398/2835] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:17:22 2022] PUT /identity/v3/projects/eafb7937222b4225b2bfbb30ddfc1950/users/41244285613d4d2ea744668b8ae8e32c/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:17:22.392266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb51b2-6f50-49be-98a2-ea4e579b948f None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:22.392758 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb51b2-6f50-49be-98a2-ea4e579b948f None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:22.393035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb51b2-6f50-49be-98a2-ea4e579b948f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:22.442641 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-53eb51b2-6f50-49be-98a2-ea4e579b948f None None] MFA Rules not processed for user `41244285613d4d2ea744668b8ae8e32c`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:22.443910 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-53eb51b2-6f50-49be-98a2-ea4e579b948f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:22.512819 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feed35e-5c6c-453e-bccf-afd90db97883 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:22.519772 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1399/2836] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:22 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:22.532555 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb51b2-6f50-49be-98a2-ea4e579b948f None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:22.533621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-53eb51b2-6f50-49be-98a2-ea4e579b948f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:22.594925 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7e70c-ef65-4c0c-9727-af6f5396cffe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:22.596724 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7e70c-ef65-4c0c-9727-af6f5396cffe service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:22.597225 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7e70c-ef65-4c0c-9727-af6f5396cffe service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:22.597582 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7e70c-ef65-4c0c-9727-af6f5396cffe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:22.619943 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:22.636996 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53eb51b2-6f50-49be-98a2-ea4e579b948f None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:22.644229 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3fe758-e51d-4e5c-9887-2a8ce2be1214 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:22.644763 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fe758-e51d-4e5c-9887-2a8ce2be1214 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:22.645103 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fe758-e51d-4e5c-9887-2a8ce2be1214 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:22.645929 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3fe758-e51d-4e5c-9887-2a8ce2be1214 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:22.655636 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3fe758-e51d-4e5c-9887-2a8ce2be1214 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:22.657193 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3fe758-e51d-4e5c-9887-2a8ce2be1214 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:22.667206 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1400/2837] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:17:22 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:17:22.668539 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ccc6e-ff1f-4177-841f-160d4602f993 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:22.668688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7e70c-ef65-4c0c-9727-af6f5396cffe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:22.668951 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ccc6e-ff1f-4177-841f-160d4602f993 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:22.669321 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ccc6e-ff1f-4177-841f-160d4602f993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:22.669984 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7e70c-ef65-4c0c-9727-af6f5396cffe service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:22.680311 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1438/2838] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:17:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:17:22.710020 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-932ccc6e-ff1f-4177-841f-160d4602f993 None None] MFA Rules not processed for user `2b62d2360a2a4e278b45d1a5c45e0cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:22.711265 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-932ccc6e-ff1f-4177-841f-160d4602f993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:22.798164 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:22.807185 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1401/2839] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:22 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:22.819090 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7e70c-ef65-4c0c-9727-af6f5396cffe service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:22.820473 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5da7e70c-ef65-4c0c-9727-af6f5396cffe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:22.979034 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7e70c-ef65-4c0c-9727-af6f5396cffe service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:22.985054 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425730e-4126-42ea-ac01-8b4f2fc8f214 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:22.985765 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425730e-4126-42ea-ac01-8b4f2fc8f214 service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:22.986049 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425730e-4126-42ea-ac01-8b4f2fc8f214 service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:22.986346 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425730e-4126-42ea-ac01-8b4f2fc8f214 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:22.988493 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5425730e-4126-42ea-ac01-8b4f2fc8f214 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:23.064030 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5425730e-4126-42ea-ac01-8b4f2fc8f214 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:23.067193 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5425730e-4126-42ea-ac01-8b4f2fc8f214 service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:23.083189 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1439/2840] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:17:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:17:28.346230 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ccc6e-ff1f-4177-841f-160d4602f993 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:28.411205 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ccc6e-ff1f-4177-841f-160d4602f993 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:28.415644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bb19e-b9f0-47b3-95b0-34a47b5c788e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:28.416275 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bb19e-b9f0-47b3-95b0-34a47b5c788e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:28.416551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bb19e-b9f0-47b3-95b0-34a47b5c788e admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:28.416819 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bb19e-b9f0-47b3-95b0-34a47b5c788e admin admin] PATH_INFO: `/v3/users/1182c1072e5b4770a2c7eb303a7e3a4e` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:28.417924 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418bb19e-b9f0-47b3-95b0-34a47b5c788e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1182c1072e5b4770a2c7eb303a7e3a4e)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:28.419852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418bb19e-b9f0-47b3-95b0-34a47b5c788e admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:28.464881 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-418bb19e-b9f0-47b3-95b0-34a47b5c788e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1182c1072e5b4770a2c7eb303a7e3a4e'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:28.465527 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-418bb19e-b9f0-47b3-95b0-34a47b5c788e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1182c1072e5b4770a2c7eb303a7e3a4e'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:28.477460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-418bb19e-b9f0-47b3-95b0-34a47b5c788e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1182c1072e5b4770a2c7eb303a7e3a4e'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:28.484523 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-418bb19e-b9f0-47b3-95b0-34a47b5c788e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1182c1072e5b4770a2c7eb303a7e3a4e'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:28.485263 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-418bb19e-b9f0-47b3-95b0-34a47b5c788e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1182c1072e5b4770a2c7eb303a7e3a4e'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:28.500705 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1402/2841] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:17:28 2022] DELETE /identity/v3/users/1182c1072e5b4770a2c7eb303a7e3a4e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:28.505190 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0899ca5-8e94-4882-a790-85399e5e3be4 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:28.505689 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0899ca5-8e94-4882-a790-85399e5e3be4 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:28.505791 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0899ca5-8e94-4882-a790-85399e5e3be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:28.524403 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-f0899ca5-8e94-4882-a790-85399e5e3be4 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:28.525565 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f0899ca5-8e94-4882-a790-85399e5e3be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:28.550300 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=100 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:28.586207 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1440/2842] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:28.595909 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bb19e-b9f0-47b3-95b0-34a47b5c788e admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:28.597337 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-418bb19e-b9f0-47b3-95b0-34a47b5c788e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:28.652778 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bb19e-b9f0-47b3-95b0-34a47b5c788e admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:28.656653 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678b494-2038-41ca-abc6-0213a1c1f4bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:28.657299 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678b494-2038-41ca-abc6-0213a1c1f4bd service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:28.657598 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678b494-2038-41ca-abc6-0213a1c1f4bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:28.657895 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678b494-2038-41ca-abc6-0213a1c1f4bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:28.668818 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678b494-2038-41ca-abc6-0213a1c1f4bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:28.670242 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2678b494-2038-41ca-abc6-0213a1c1f4bd service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:28.680562 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1403/2843] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:17:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:17:28.956937 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0899ca5-8e94-4882-a790-85399e5e3be4 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:28.959581 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f0899ca5-8e94-4882-a790-85399e5e3be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:28.982857 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0899ca5-8e94-4882-a790-85399e5e3be4 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:28.993567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098ff63-e264-4b16-ad57-7d4b2f99a90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:28.999117 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098ff63-e264-4b16-ad57-7d4b2f99a90c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:28.999575 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098ff63-e264-4b16-ad57-7d4b2f99a90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:29.000004 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098ff63-e264-4b16-ad57-7d4b2f99a90c admin admin] PATH_INFO: `/v3/projects/83d70cb638b84a47bef3eec9df2bf372` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:29.001876 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098ff63-e264-4b16-ad57-7d4b2f99a90c admin admin] RBAC: Authorizing `identity:delete_project(project_id=83d70cb638b84a47bef3eec9df2bf372)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:29.004375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098ff63-e264-4b16-ad57-7d4b2f99a90c admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:29.085279 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e098ff63-e264-4b16-ad57-7d4b2f99a90c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83d70cb638b84a47bef3eec9df2bf372', 'request_id': 'req-e098ff63-e264-4b16-ad57-7d4b2f99a90c'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:29.092899 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e098ff63-e264-4b16-ad57-7d4b2f99a90c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83d70cb638b84a47bef3eec9df2bf372', 'request_id': 'req-e098ff63-e264-4b16-ad57-7d4b2f99a90c'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:29.099013 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e098ff63-e264-4b16-ad57-7d4b2f99a90c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83d70cb638b84a47bef3eec9df2bf372', 'request_id': 'req-e098ff63-e264-4b16-ad57-7d4b2f99a90c'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:29.112283 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e098ff63-e264-4b16-ad57-7d4b2f99a90c admin admin] The token cache is being invalidate because project 83d70cb638b84a47bef3eec9df2bf372 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:17:29.112668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e098ff63-e264-4b16-ad57-7d4b2f99a90c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:29.113315 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-e098ff63-e264-4b16-ad57-7d4b2f99a90c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:29.114837 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1441/2844] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:17:28 2022] DELETE /identity/v3/projects/83d70cb638b84a47bef3eec9df2bf372 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:29.119858 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef2e50-76d3-4252-8e3b-a3371c5b8a0f None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:29.120136 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef2e50-76d3-4252-8e3b-a3371c5b8a0f None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:29.120400 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef2e50-76d3-4252-8e3b-a3371c5b8a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:29.140047 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-ebef2e50-76d3-4252-8e3b-a3371c5b8a0f None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:29.141262 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ebef2e50-76d3-4252-8e3b-a3371c5b8a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:29.158569 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:29.200747 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1404/2845] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:29.210496 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac6208-ec28-441e-88d1-ad96156b8a18 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:29.210788 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac6208-ec28-441e-88d1-ad96156b8a18 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:29.211063 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac6208-ec28-441e-88d1-ad96156b8a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:29.230359 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-f9ac6208-ec28-441e-88d1-ad96156b8a18 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:29.231644 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f9ac6208-ec28-441e-88d1-ad96156b8a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:29.239615 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1442/2846] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:29.243894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef2e50-76d3-4252-8e3b-a3371c5b8a0f None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:29.254035 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef2e50-76d3-4252-8e3b-a3371c5b8a0f None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:29.259532 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb9988-4495-4a0f-86d7-33f0bb4971ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:29.260149 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb9988-4495-4a0f-86d7-33f0bb4971ae admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:29.260432 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb9988-4495-4a0f-86d7-33f0bb4971ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:29.260706 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb9988-4495-4a0f-86d7-33f0bb4971ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:29.261369 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb9988-4495-4a0f-86d7-33f0bb4971ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81390) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:29.261674 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb9988-4495-4a0f-86d7-33f0bb4971ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:29.263445 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb9988-4495-4a0f-86d7-33f0bb4971ae admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:29.277433 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1405/2847] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:17:29.281778 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac6208-ec28-441e-88d1-ad96156b8a18 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:29.291130 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac6208-ec28-441e-88d1-ad96156b8a18 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:29.296258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42dbcd9-0e27-46ba-8279-b5372a4a1f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:29.296850 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dbcd9-0e27-46ba-8279-b5372a4a1f21 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:29.297128 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dbcd9-0e27-46ba-8279-b5372a4a1f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:29.297398 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dbcd9-0e27-46ba-8279-b5372a4a1f21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:29.298045 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42dbcd9-0e27-46ba-8279-b5372a4a1f21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:29.300191 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42dbcd9-0e27-46ba-8279-b5372a4a1f21 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:29.327456 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1443/2848] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:17:29 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:17:29.331721 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb9988-4495-4a0f-86d7-33f0bb4971ae admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:29.342518 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb9988-4495-4a0f-86d7-33f0bb4971ae admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:29.402801 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b6a52-631d-4880-9ae9-391d2cdf18a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:29.403464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b6a52-631d-4880-9ae9-391d2cdf18a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:29.403696 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b6a52-631d-4880-9ae9-391d2cdf18a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:29.403971 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b6a52-631d-4880-9ae9-391d2cdf18a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:29.404716 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b6a52-631d-4880-9ae9-391d2cdf18a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:29.406647 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2b6a52-631d-4880-9ae9-391d2cdf18a7 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:29.428306 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1406/2849] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:17:29 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:17:29.432459 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42dbcd9-0e27-46ba-8279-b5372a4a1f21 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:29.444309 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42dbcd9-0e27-46ba-8279-b5372a4a1f21 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:29.449279 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093b98b-84cd-4a7f-8ae0-8f041c7cf83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:29.449884 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093b98b-84cd-4a7f-8ae0-8f041c7cf83d admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:29.450165 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093b98b-84cd-4a7f-8ae0-8f041c7cf83d admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:29.450437 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093b98b-84cd-4a7f-8ae0-8f041c7cf83d admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:29.451130 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e093b98b-84cd-4a7f-8ae0-8f041c7cf83d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:29.452734 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e093b98b-84cd-4a7f-8ae0-8f041c7cf83d admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:29.463394 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1444/2850] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:17:29.467464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b6a52-631d-4880-9ae9-391d2cdf18a7 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:29.477427 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b6a52-631d-4880-9ae9-391d2cdf18a7 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:29.482046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a528c66-bd79-4ef3-8d7b-64cb0a3969d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:29.482639 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a528c66-bd79-4ef3-8d7b-64cb0a3969d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:29.482917 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a528c66-bd79-4ef3-8d7b-64cb0a3969d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:29.483218 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a528c66-bd79-4ef3-8d7b-64cb0a3969d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:29.483852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a528c66-bd79-4ef3-8d7b-64cb0a3969d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:29.485426 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a528c66-bd79-4ef3-8d7b-64cb0a3969d8 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:29.496277 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1407/2851] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:17:29.500344 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093b98b-84cd-4a7f-8ae0-8f041c7cf83d admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:29.510720 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093b98b-84cd-4a7f-8ae0-8f041c7cf83d admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:29.515432 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb8a6d-1e75-4f5a-addc-03efc68686cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:29.516077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb8a6d-1e75-4f5a-addc-03efc68686cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:29.516358 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb8a6d-1e75-4f5a-addc-03efc68686cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:29.516633 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cb8a6d-1e75-4f5a-addc-03efc68686cd admin admin] PATH_INFO: `/v3/projects/0451862b021543bfb3c94066579d5c22/users/3900fa665a334c12b06afa5606e0035a/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:29.541226 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb8a6d-1e75-4f5a-addc-03efc68686cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0451862b021543bfb3c94066579d5c22, user_id=3900fa665a334c12b06afa5606e0035a, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:29.543959 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cb8a6d-1e75-4f5a-addc-03efc68686cd admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:29.556477 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1445/2852] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:17:29 2022] PUT /identity/v3/projects/0451862b021543bfb3c94066579d5c22/users/3900fa665a334c12b06afa5606e0035a/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:29.566808 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f847c-26eb-44b3-9fcc-223fae837034 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:29.567629 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f847c-26eb-44b3-9fcc-223fae837034 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:29.567925 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f847c-26eb-44b3-9fcc-223fae837034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:29.595130 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-9e6f847c-26eb-44b3-9fcc-223fae837034 None None] MFA Rules not processed for user `3900fa665a334c12b06afa5606e0035a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:29.596366 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-9e6f847c-26eb-44b3-9fcc-223fae837034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:29.660793 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1408/2853] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:17:29.665237 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb8a6d-1e75-4f5a-addc-03efc68686cd admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:29.721921 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:29.733925 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cb8a6d-1e75-4f5a-addc-03efc68686cd admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:29.739190 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98254dcc-3daf-4ba5-8c24-404ecc8a1385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:29.739793 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98254dcc-3daf-4ba5-8c24-404ecc8a1385 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:29.740065 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98254dcc-3daf-4ba5-8c24-404ecc8a1385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:29.740328 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98254dcc-3daf-4ba5-8c24-404ecc8a1385 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:29.740955 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98254dcc-3daf-4ba5-8c24-404ecc8a1385 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:29.742831 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98254dcc-3daf-4ba5-8c24-404ecc8a1385 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:29.768256 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1446/2854] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:17:29 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:17:29.772333 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f847c-26eb-44b3-9fcc-223fae837034 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:29.783875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f847c-26eb-44b3-9fcc-223fae837034 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:29.805839 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:29.849355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570443a-75b2-4021-b2ef-0714aa94fe32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:29.849955 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570443a-75b2-4021-b2ef-0714aa94fe32 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:29.850229 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570443a-75b2-4021-b2ef-0714aa94fe32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:29.850496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570443a-75b2-4021-b2ef-0714aa94fe32 admin admin] PATH_INFO: `/v3/users` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:29.851166 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570443a-75b2-4021-b2ef-0714aa94fe32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:29.853076 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570443a-75b2-4021-b2ef-0714aa94fe32 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:29.876474 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1409/2855] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:17:29 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:17:29.880458 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98254dcc-3daf-4ba5-8c24-404ecc8a1385 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:29.890659 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98254dcc-3daf-4ba5-8c24-404ecc8a1385 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:29.895171 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab37ac3-1eb3-47c9-8e70-577e11ad0897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:29.895752 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab37ac3-1eb3-47c9-8e70-577e11ad0897 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:29.896026 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab37ac3-1eb3-47c9-8e70-577e11ad0897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:29.896296 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab37ac3-1eb3-47c9-8e70-577e11ad0897 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:29.896924 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab37ac3-1eb3-47c9-8e70-577e11ad0897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:29.898497 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab37ac3-1eb3-47c9-8e70-577e11ad0897 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:29.908738 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1447/2856] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:17:29.913212 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570443a-75b2-4021-b2ef-0714aa94fe32 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:29.922936 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570443a-75b2-4021-b2ef-0714aa94fe32 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:29.928092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5032fb2-ff5f-4575-acf9-1a017e76e064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:29.928679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5032fb2-ff5f-4575-acf9-1a017e76e064 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:29.928957 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5032fb2-ff5f-4575-acf9-1a017e76e064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:29.929226 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5032fb2-ff5f-4575-acf9-1a017e76e064 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:29.929882 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5032fb2-ff5f-4575-acf9-1a017e76e064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:29.931478 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5032fb2-ff5f-4575-acf9-1a017e76e064 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:29.941212 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1410/2857] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:17:29.945130 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab37ac3-1eb3-47c9-8e70-577e11ad0897 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:29.954645 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab37ac3-1eb3-47c9-8e70-577e11ad0897 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:29.960557 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fbb1f-0e8d-4c2f-9151-f3cb0e6ec2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:29.961275 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fbb1f-0e8d-4c2f-9151-f3cb0e6ec2e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:29.961552 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fbb1f-0e8d-4c2f-9151-f3cb0e6ec2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:29.961834 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fbb1f-0e8d-4c2f-9151-f3cb0e6ec2e2 admin admin] PATH_INFO: `/v3/projects/155f8e5c83c14d5f8ae6c45a9fe8cf2d/users/33c61f51d0304b518dc4b5b946306b4a/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:29.978559 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fbb1f-0e8d-4c2f-9151-f3cb0e6ec2e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=155f8e5c83c14d5f8ae6c45a9fe8cf2d, user_id=33c61f51d0304b518dc4b5b946306b4a, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:29.981294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fbb1f-0e8d-4c2f-9151-f3cb0e6ec2e2 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:29.994263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1448/2858] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:17:29 2022] PUT /identity/v3/projects/155f8e5c83c14d5f8ae6c45a9fe8cf2d/users/33c61f51d0304b518dc4b5b946306b4a/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:30.005324 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19af4b-d825-45bf-ade6-c061110088bd None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:30.005614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19af4b-d825-45bf-ade6-c061110088bd None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:30.005888 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19af4b-d825-45bf-ade6-c061110088bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:30.034367 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-3e19af4b-d825-45bf-ade6-c061110088bd None None] MFA Rules not processed for user `33c61f51d0304b518dc4b5b946306b4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:30.036121 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-3e19af4b-d825-45bf-ade6-c061110088bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:30.106464 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1411/2859] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:30.110965 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fbb1f-0e8d-4c2f-9151-f3cb0e6ec2e2 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:30.124239 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19af4b-d825-45bf-ade6-c061110088bd None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:30.125694 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-3e19af4b-d825-45bf-ade6-c061110088bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:30.179419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fbb1f-0e8d-4c2f-9151-f3cb0e6ec2e2 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:30.184358 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c8c64-875d-4f34-8307-391f8e52e5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:30.185045 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c8c64-875d-4f34-8307-391f8e52e5d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:30.185332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c8c64-875d-4f34-8307-391f8e52e5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:30.185608 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c8c64-875d-4f34-8307-391f8e52e5d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:30.186272 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3c8c64-875d-4f34-8307-391f8e52e5d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:30.188213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3c8c64-875d-4f34-8307-391f8e52e5d6 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:30.195718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19af4b-d825-45bf-ade6-c061110088bd None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:30.200300 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d8018-8ef4-4d7d-aef6-0b990248eb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:30.200928 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d8018-8ef4-4d7d-aef6-0b990248eb48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:30.201200 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d8018-8ef4-4d7d-aef6-0b990248eb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:30.201469 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d8018-8ef4-4d7d-aef6-0b990248eb48 admin admin] PATH_INFO: `/v3/users/41244285613d4d2ea744668b8ae8e32c` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:30.202591 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d8018-8ef4-4d7d-aef6-0b990248eb48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41244285613d4d2ea744668b8ae8e32c)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:30.204489 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d8018-8ef4-4d7d-aef6-0b990248eb48 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:30.216471 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1449/2860] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:17:30 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:17:30.220331 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c8c64-875d-4f34-8307-391f8e52e5d6 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:30.239136 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:30.259537 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c69d8018-8ef4-4d7d-aef6-0b990248eb48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41244285613d4d2ea744668b8ae8e32c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:30.260720 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c69d8018-8ef4-4d7d-aef6-0b990248eb48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41244285613d4d2ea744668b8ae8e32c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:30.274551 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c69d8018-8ef4-4d7d-aef6-0b990248eb48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41244285613d4d2ea744668b8ae8e32c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:30.281631 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c69d8018-8ef4-4d7d-aef6-0b990248eb48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41244285613d4d2ea744668b8ae8e32c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:30.282291 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-c69d8018-8ef4-4d7d-aef6-0b990248eb48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41244285613d4d2ea744668b8ae8e32c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:30.291298 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c8c64-875d-4f34-8307-391f8e52e5d6 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:30.296184 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1412/2861] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:17:30 2022] DELETE /identity/v3/users/41244285613d4d2ea744668b8ae8e32c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:17:30.298266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9e1eb-895c-46cd-8fdc-1483f2986472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:30.299193 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9e1eb-895c-46cd-8fdc-1483f2986472 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:30.299612 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9e1eb-895c-46cd-8fdc-1483f2986472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:30.300025 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c9e1eb-895c-46cd-8fdc-1483f2986472 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:30.300813 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0da43-a49b-4b4a-be6d-c3bb6f73f55c None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:30.301104 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0da43-a49b-4b4a-be6d-c3bb6f73f55c None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:30.301290 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9e1eb-895c-46cd-8fdc-1483f2986472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:30.301394 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda0da43-a49b-4b4a-be6d-c3bb6f73f55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:30.304294 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c9e1eb-895c-46cd-8fdc-1483f2986472 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:30.321387 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-bda0da43-a49b-4b4a-be6d-c3bb6f73f55c None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:30.323143 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-bda0da43-a49b-4b4a-be6d-c3bb6f73f55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:30.339463 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1450/2862] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:17:30 2022] POST /identity/v3/users => generated 299 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:17:30.343331 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9e1eb-895c-46cd-8fdc-1483f2986472 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:30.345103 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-19c9e1eb-895c-46cd-8fdc-1483f2986472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:30.362694 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c9e1eb-895c-46cd-8fdc-1483f2986472 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:30.374487 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:30.393644 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1413/2863] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:30.404659 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0da43-a49b-4b4a-be6d-c3bb6f73f55c None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:30.406229 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-bda0da43-a49b-4b4a-be6d-c3bb6f73f55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:30.428042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e025a-571d-4a4c-b0a4-e9db7b3ce478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:30.428631 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e025a-571d-4a4c-b0a4-e9db7b3ce478 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:30.428904 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e025a-571d-4a4c-b0a4-e9db7b3ce478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:30.429172 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2e025a-571d-4a4c-b0a4-e9db7b3ce478 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:30.429870 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e025a-571d-4a4c-b0a4-e9db7b3ce478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:30.431429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2e025a-571d-4a4c-b0a4-e9db7b3ce478 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:30.441151 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1451/2864] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:17:30.445276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e025a-571d-4a4c-b0a4-e9db7b3ce478 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:30.456001 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2e025a-571d-4a4c-b0a4-e9db7b3ce478 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:30.461503 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773652a9-f225-4e5a-b788-db8902ebdbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:30.462126 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773652a9-f225-4e5a-b788-db8902ebdbfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:30.462430 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773652a9-f225-4e5a-b788-db8902ebdbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:30.462697 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773652a9-f225-4e5a-b788-db8902ebdbfd admin admin] PATH_INFO: `/v3/projects/2ba67c6da87545f7847cd4afbf0d32cd/users/f03a75f64b384c88af50833884fde905/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:30.465750 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda0da43-a49b-4b4a-be6d-c3bb6f73f55c None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:30.469517 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58b9f1-be4c-45de-bc19-5905cda4bf22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:30.470157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58b9f1-be4c-45de-bc19-5905cda4bf22 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:30.470438 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58b9f1-be4c-45de-bc19-5905cda4bf22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:30.470734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf58b9f1-be4c-45de-bc19-5905cda4bf22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:30.481138 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773652a9-f225-4e5a-b788-db8902ebdbfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba67c6da87545f7847cd4afbf0d32cd, user_id=f03a75f64b384c88af50833884fde905, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:30.481589 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58b9f1-be4c-45de-bc19-5905cda4bf22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:30.483168 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf58b9f1-be4c-45de-bc19-5905cda4bf22 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:30.483948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773652a9-f225-4e5a-b788-db8902ebdbfd admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:30.494138 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1414/2865] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:17:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:17:30.499495 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1452/2866] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:17:30 2022] PUT /identity/v3/projects/2ba67c6da87545f7847cd4afbf0d32cd/users/f03a75f64b384c88af50833884fde905/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:17:30.503359 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58b9f1-be4c-45de-bc19-5905cda4bf22 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:30.513945 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf58b9f1-be4c-45de-bc19-5905cda4bf22 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:30.560233 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:30.574847 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6ea0e-5cd7-455d-b2a5-844388079204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:30.575486 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6ea0e-5cd7-455d-b2a5-844388079204 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:30.576083 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6ea0e-5cd7-455d-b2a5-844388079204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:30.576620 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6ea0e-5cd7-455d-b2a5-844388079204 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:30.577378 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6ea0e-5cd7-455d-b2a5-844388079204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:30.579278 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6ea0e-5cd7-455d-b2a5-844388079204 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:30.591299 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1415/2867] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:17:30.596064 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773652a9-f225-4e5a-b788-db8902ebdbfd admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:30.613902 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773652a9-f225-4e5a-b788-db8902ebdbfd admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:30.619450 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35aeaf4-a7eb-43aa-ae0a-bf031ac1f5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:30.620354 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35aeaf4-a7eb-43aa-ae0a-bf031ac1f5d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:30.620656 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35aeaf4-a7eb-43aa-ae0a-bf031ac1f5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:30.621093 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35aeaf4-a7eb-43aa-ae0a-bf031ac1f5d7 admin admin] PATH_INFO: `/v3/projects/2ba67c6da87545f7847cd4afbf0d32cd/users/f03a75f64b384c88af50833884fde905/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:30.623649 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35aeaf4-a7eb-43aa-ae0a-bf031ac1f5d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba67c6da87545f7847cd4afbf0d32cd, user_id=f03a75f64b384c88af50833884fde905, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:30.626372 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35aeaf4-a7eb-43aa-ae0a-bf031ac1f5d7 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:30.637451 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1453/2868] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:17:30 2022] PUT /identity/v3/projects/2ba67c6da87545f7847cd4afbf0d32cd/users/f03a75f64b384c88af50833884fde905/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:30.720587 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21d4ec-2978-42e2-83b4-04d570bb88b3 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:30.720830 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21d4ec-2978-42e2-83b4-04d570bb88b3 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:30.721061 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21d4ec-2978-42e2-83b4-04d570bb88b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:30.750074 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-de21d4ec-2978-42e2-83b4-04d570bb88b3 None None] MFA Rules not processed for user `f03a75f64b384c88af50833884fde905`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:30.751452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-de21d4ec-2978-42e2-83b4-04d570bb88b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:30.803257 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35aeaf4-a7eb-43aa-ae0a-bf031ac1f5d7 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:30.805249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-e35aeaf4-a7eb-43aa-ae0a-bf031ac1f5d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:30.834998 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1416/2869] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:30.846677 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21d4ec-2978-42e2-83b4-04d570bb88b3 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:30.848410 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-de21d4ec-2978-42e2-83b4-04d570bb88b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:30.874685 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35aeaf4-a7eb-43aa-ae0a-bf031ac1f5d7 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:30.880088 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069829a-fb36-4d6d-8449-a0eb2f354ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:30.880837 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069829a-fb36-4d6d-8449-a0eb2f354ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:30.881004 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069829a-fb36-4d6d-8449-a0eb2f354ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:30.881274 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069829a-fb36-4d6d-8449-a0eb2f354ac5 admin admin] PATH_INFO: `/v3/projects/eafb7937222b4225b2bfbb30ddfc1950` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:30.882625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069829a-fb36-4d6d-8449-a0eb2f354ac5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eafb7937222b4225b2bfbb30ddfc1950)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:30.884583 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069829a-fb36-4d6d-8449-a0eb2f354ac5 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:30.910353 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21d4ec-2978-42e2-83b4-04d570bb88b3 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:30.914502 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73149f-f5ed-4af8-835e-ab1564b5d4be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:30.915351 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73149f-f5ed-4af8-835e-ab1564b5d4be service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:30.915634 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73149f-f5ed-4af8-835e-ab1564b5d4be service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:30.915894 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed73149f-f5ed-4af8-835e-ab1564b5d4be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:30.917720 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-ed73149f-f5ed-4af8-835e-ab1564b5d4be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:30.933912 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-0069829a-fb36-4d6d-8449-a0eb2f354ac5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eafb7937222b4225b2bfbb30ddfc1950', 'request_id': 'req-0069829a-fb36-4d6d-8449-a0eb2f354ac5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:30.941493 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-0069829a-fb36-4d6d-8449-a0eb2f354ac5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eafb7937222b4225b2bfbb30ddfc1950', 'request_id': 'req-0069829a-fb36-4d6d-8449-a0eb2f354ac5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:30.947753 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-0069829a-fb36-4d6d-8449-a0eb2f354ac5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eafb7937222b4225b2bfbb30ddfc1950', 'request_id': 'req-0069829a-fb36-4d6d-8449-a0eb2f354ac5'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:30.959658 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-0069829a-fb36-4d6d-8449-a0eb2f354ac5 admin admin] The token cache is being invalidate because project eafb7937222b4225b2bfbb30ddfc1950 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:17:30.960194 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-0069829a-fb36-4d6d-8449-a0eb2f354ac5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:30.960879 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-0069829a-fb36-4d6d-8449-a0eb2f354ac5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:30.965226 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1454/2870] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:17:30 2022] DELETE /identity/v3/projects/eafb7937222b4225b2bfbb30ddfc1950 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:30.983946 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed73149f-f5ed-4af8-835e-ab1564b5d4be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:30.985377 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed73149f-f5ed-4af8-835e-ab1564b5d4be service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:30.996336 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1417/2871] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:17:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:17:32.541316 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069829a-fb36-4d6d-8449-a0eb2f354ac5 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:32.542721 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-0069829a-fb36-4d6d-8449-a0eb2f354ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:32.549426 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:32.603329 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069829a-fb36-4d6d-8449-a0eb2f354ac5 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:32.607117 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee780a08-fbf6-442d-8de3-23afc0921232 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:32.607721 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee780a08-fbf6-442d-8de3-23afc0921232 service cinder] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:32.608002 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee780a08-fbf6-442d-8de3-23afc0921232 service cinder] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:32.608272 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee780a08-fbf6-442d-8de3-23afc0921232 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:32.610019 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-ee780a08-fbf6-442d-8de3-23afc0921232 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:32.670990 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee780a08-fbf6-442d-8de3-23afc0921232 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:32.672447 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee780a08-fbf6-442d-8de3-23afc0921232 service cinder] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:32.681206 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1455/2872] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:17:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:17:33.214748 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73149f-f5ed-4af8-835e-ab1564b5d4be service cinder] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:33.225447 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed73149f-f5ed-4af8-835e-ab1564b5d4be service cinder] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:33.229010 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680d4a5-5dbd-4bb5-8db0-c13f7b098638 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8370135a78f414f9184c883ccb92767', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:33.229596 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680d4a5-5dbd-4bb5-8db0-c13f7b098638 service cinder] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:33.229873 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680d4a5-5dbd-4bb5-8db0-c13f7b098638 service cinder] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:33.230139 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680d4a5-5dbd-4bb5-8db0-c13f7b098638 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:33.231895 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6680d4a5-5dbd-4bb5-8db0-c13f7b098638 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:33.255052 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:33.296591 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680d4a5-5dbd-4bb5-8db0-c13f7b098638 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:33.298028 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680d4a5-5dbd-4bb5-8db0-c13f7b098638 service cinder] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:33.307964 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1418/2873] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:17:33 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:17:37.166516 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee780a08-fbf6-442d-8de3-23afc0921232 service cinder] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:37.167946 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-ee780a08-fbf6-442d-8de3-23afc0921232 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:37.231923 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee780a08-fbf6-442d-8de3-23afc0921232 service cinder] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:37.237520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762b3a1-77f1-4c88-af65-d072fb97af8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:37.238159 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762b3a1-77f1-4c88-af65-d072fb97af8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:37.238440 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762b3a1-77f1-4c88-af65-d072fb97af8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:37.239267 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762b3a1-77f1-4c88-af65-d072fb97af8a admin admin] PATH_INFO: `/v3/users/3900fa665a334c12b06afa5606e0035a` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:37.240455 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762b3a1-77f1-4c88-af65-d072fb97af8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3900fa665a334c12b06afa5606e0035a)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:37.242405 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762b3a1-77f1-4c88-af65-d072fb97af8a admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:37.291826 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5762b3a1-77f1-4c88-af65-d072fb97af8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3900fa665a334c12b06afa5606e0035a'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:37.298795 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5762b3a1-77f1-4c88-af65-d072fb97af8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3900fa665a334c12b06afa5606e0035a'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:37.303601 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:37.308090 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5762b3a1-77f1-4c88-af65-d072fb97af8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3900fa665a334c12b06afa5606e0035a'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:37.309042 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5762b3a1-77f1-4c88-af65-d072fb97af8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3900fa665a334c12b06afa5606e0035a'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:37.310132 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5762b3a1-77f1-4c88-af65-d072fb97af8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3900fa665a334c12b06afa5606e0035a'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:37.328266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1456/2874] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:17:37 2022] DELETE /identity/v3/users/3900fa665a334c12b06afa5606e0035a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:37.333548 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c3a8d-eede-4333-bbed-31937a1b9686 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:37.334097 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c3a8d-eede-4333-bbed-31937a1b9686 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:37.334680 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c3a8d-eede-4333-bbed-31937a1b9686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:37.355626 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-6e1c3a8d-eede-4333-bbed-31937a1b9686 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:37.356818 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6e1c3a8d-eede-4333-bbed-31937a1b9686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:37.415953 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1419/2875] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:37.432429 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762b3a1-77f1-4c88-af65-d072fb97af8a admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:37.434646 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5762b3a1-77f1-4c88-af65-d072fb97af8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:37.491868 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762b3a1-77f1-4c88-af65-d072fb97af8a admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:37.495644 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8445609-2bb4-48ce-a496-8fda7d2e0466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:37.496246 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8445609-2bb4-48ce-a496-8fda7d2e0466 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:37.496517 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8445609-2bb4-48ce-a496-8fda7d2e0466 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:37.496781 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8445609-2bb4-48ce-a496-8fda7d2e0466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:37.506049 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8445609-2bb4-48ce-a496-8fda7d2e0466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:37.507509 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8445609-2bb4-48ce-a496-8fda7d2e0466 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:37.517688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1457/2876] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:17:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:17:37.782966 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c3a8d-eede-4333-bbed-31937a1b9686 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:37.784419 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-6e1c3a8d-eede-4333-bbed-31937a1b9686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:37.798070 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c3a8d-eede-4333-bbed-31937a1b9686 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:37.802751 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8388e-b208-43be-8fd6-382b06ef8484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:37.803425 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8388e-b208-43be-8fd6-382b06ef8484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:37.803743 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8388e-b208-43be-8fd6-382b06ef8484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:37.803968 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8388e-b208-43be-8fd6-382b06ef8484 admin admin] PATH_INFO: `/v3/projects/0451862b021543bfb3c94066579d5c22` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:37.805087 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8388e-b208-43be-8fd6-382b06ef8484 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0451862b021543bfb3c94066579d5c22)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:37.807015 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8388e-b208-43be-8fd6-382b06ef8484 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:37.857143 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-78a8388e-b208-43be-8fd6-382b06ef8484 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0451862b021543bfb3c94066579d5c22', 'request_id': 'req-78a8388e-b208-43be-8fd6-382b06ef8484'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:37.863474 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-78a8388e-b208-43be-8fd6-382b06ef8484 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0451862b021543bfb3c94066579d5c22', 'request_id': 'req-78a8388e-b208-43be-8fd6-382b06ef8484'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:37.869221 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-78a8388e-b208-43be-8fd6-382b06ef8484 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0451862b021543bfb3c94066579d5c22', 'request_id': 'req-78a8388e-b208-43be-8fd6-382b06ef8484'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:37.881974 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-78a8388e-b208-43be-8fd6-382b06ef8484 admin admin] The token cache is being invalidate because project 0451862b021543bfb3c94066579d5c22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:17:37.882266 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-78a8388e-b208-43be-8fd6-382b06ef8484 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:37.882890 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-78a8388e-b208-43be-8fd6-382b06ef8484 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:37.884453 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1420/2877] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:17:37 2022] DELETE /identity/v3/projects/0451862b021543bfb3c94066579d5c22 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:37.888548 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8445609-2bb4-48ce-a496-8fda7d2e0466 service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:37.898389 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8445609-2bb4-48ce-a496-8fda7d2e0466 service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:37.958351 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c16cb-b1a1-4272-bb86-966adeab5bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:37.958971 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c16cb-b1a1-4272-bb86-966adeab5bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:37.959274 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c16cb-b1a1-4272-bb86-966adeab5bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:37.959795 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4c16cb-b1a1-4272-bb86-966adeab5bea admin admin] PATH_INFO: `/v3/users/33c61f51d0304b518dc4b5b946306b4a` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:37.960893 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4c16cb-b1a1-4272-bb86-966adeab5bea admin admin] RBAC: Authorizing `identity:delete_user(user_id=33c61f51d0304b518dc4b5b946306b4a)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:37.962788 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4c16cb-b1a1-4272-bb86-966adeab5bea admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:38.011252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5c4c16cb-b1a1-4272-bb86-966adeab5bea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33c61f51d0304b518dc4b5b946306b4a'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:38.018801 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5c4c16cb-b1a1-4272-bb86-966adeab5bea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33c61f51d0304b518dc4b5b946306b4a'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:38.026295 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5c4c16cb-b1a1-4272-bb86-966adeab5bea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33c61f51d0304b518dc4b5b946306b4a'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:38.027021 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5c4c16cb-b1a1-4272-bb86-966adeab5bea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33c61f51d0304b518dc4b5b946306b4a'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:38.027962 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-5c4c16cb-b1a1-4272-bb86-966adeab5bea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33c61f51d0304b518dc4b5b946306b4a'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:38.043459 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:38.048093 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1458/2878] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:17:37 2022] DELETE /identity/v3/users/33c61f51d0304b518dc4b5b946306b4a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:38.318941 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8388e-b208-43be-8fd6-382b06ef8484 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:38.321373 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-78a8388e-b208-43be-8fd6-382b06ef8484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:38.342836 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8388e-b208-43be-8fd6-382b06ef8484 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:38.390510 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:38.407765 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b663501-bf59-45d2-888a-51621fbd552a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:38.408441 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b663501-bf59-45d2-888a-51621fbd552a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:38.408714 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b663501-bf59-45d2-888a-51621fbd552a admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:38.408981 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b663501-bf59-45d2-888a-51621fbd552a admin admin] PATH_INFO: `/v3/projects/155f8e5c83c14d5f8ae6c45a9fe8cf2d` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:38.410081 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b663501-bf59-45d2-888a-51621fbd552a admin admin] RBAC: Authorizing `identity:delete_project(project_id=155f8e5c83c14d5f8ae6c45a9fe8cf2d)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:38.412499 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b663501-bf59-45d2-888a-51621fbd552a admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:38.459632 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-1b663501-bf59-45d2-888a-51621fbd552a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '155f8e5c83c14d5f8ae6c45a9fe8cf2d', 'request_id': 'req-1b663501-bf59-45d2-888a-51621fbd552a'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:38.465980 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-1b663501-bf59-45d2-888a-51621fbd552a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '155f8e5c83c14d5f8ae6c45a9fe8cf2d', 'request_id': 'req-1b663501-bf59-45d2-888a-51621fbd552a'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:38.476265 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-1b663501-bf59-45d2-888a-51621fbd552a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '155f8e5c83c14d5f8ae6c45a9fe8cf2d', 'request_id': 'req-1b663501-bf59-45d2-888a-51621fbd552a'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:38.488441 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-1b663501-bf59-45d2-888a-51621fbd552a admin admin] The token cache is being invalidate because project 155f8e5c83c14d5f8ae6c45a9fe8cf2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:17:38.488730 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-1b663501-bf59-45d2-888a-51621fbd552a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:38.489341 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-1b663501-bf59-45d2-888a-51621fbd552a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:38.490801 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1421/2879] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:17:38 2022] DELETE /identity/v3/projects/155f8e5c83c14d5f8ae6c45a9fe8cf2d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:38.494895 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c16cb-b1a1-4272-bb86-966adeab5bea admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:38.496314 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-5c4c16cb-b1a1-4272-bb86-966adeab5bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:38.508756 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4c16cb-b1a1-4272-bb86-966adeab5bea admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:38.563407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4720fc5f-6ae5-451e-ae84-860fd6404060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:38.564038 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720fc5f-6ae5-451e-ae84-860fd6404060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:38.564316 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720fc5f-6ae5-451e-ae84-860fd6404060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:38.564587 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4720fc5f-6ae5-451e-ae84-860fd6404060 admin admin] PATH_INFO: `/v3/users/f03a75f64b384c88af50833884fde905` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:38.565684 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720fc5f-6ae5-451e-ae84-860fd6404060 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f03a75f64b384c88af50833884fde905)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:38.567668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4720fc5f-6ae5-451e-ae84-860fd6404060 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:38.612761 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-4720fc5f-6ae5-451e-ae84-860fd6404060 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f03a75f64b384c88af50833884fde905'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:38.619476 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-4720fc5f-6ae5-451e-ae84-860fd6404060 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f03a75f64b384c88af50833884fde905'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:38.626499 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-4720fc5f-6ae5-451e-ae84-860fd6404060 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f03a75f64b384c88af50833884fde905'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:38.627147 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-4720fc5f-6ae5-451e-ae84-860fd6404060 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f03a75f64b384c88af50833884fde905'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:38.627745 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-4720fc5f-6ae5-451e-ae84-860fd6404060 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f03a75f64b384c88af50833884fde905'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:38.644419 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1459/2880] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:17:38 2022] DELETE /identity/v3/users/f03a75f64b384c88af50833884fde905 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:38.906910 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b663501-bf59-45d2-888a-51621fbd552a admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:38.908361 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1b663501-bf59-45d2-888a-51621fbd552a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:38.921023 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b663501-bf59-45d2-888a-51621fbd552a admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:38.977460 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74056b4-86f3-487d-a0be-bdc0c261f56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:38.978094 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74056b4-86f3-487d-a0be-bdc0c261f56c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:38.978375 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74056b4-86f3-487d-a0be-bdc0c261f56c admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:38.978645 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74056b4-86f3-487d-a0be-bdc0c261f56c admin admin] PATH_INFO: `/v3/projects/2ba67c6da87545f7847cd4afbf0d32cd` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:38.979783 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74056b4-86f3-487d-a0be-bdc0c261f56c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ba67c6da87545f7847cd4afbf0d32cd)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:38.981665 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74056b4-86f3-487d-a0be-bdc0c261f56c admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:39.026045 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f74056b4-86f3-487d-a0be-bdc0c261f56c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ba67c6da87545f7847cd4afbf0d32cd', 'request_id': 'req-f74056b4-86f3-487d-a0be-bdc0c261f56c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:39.031925 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f74056b4-86f3-487d-a0be-bdc0c261f56c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ba67c6da87545f7847cd4afbf0d32cd', 'request_id': 'req-f74056b4-86f3-487d-a0be-bdc0c261f56c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:39.037219 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f74056b4-86f3-487d-a0be-bdc0c261f56c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ba67c6da87545f7847cd4afbf0d32cd', 'request_id': 'req-f74056b4-86f3-487d-a0be-bdc0c261f56c'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:39.049148 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f74056b4-86f3-487d-a0be-bdc0c261f56c admin admin] The token cache is being invalidate because project 2ba67c6da87545f7847cd4afbf0d32cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:17:39.049436 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f74056b4-86f3-487d-a0be-bdc0c261f56c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:39.050072 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-f74056b4-86f3-487d-a0be-bdc0c261f56c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:17:39.051710 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1422/2881] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:17:38 2022] DELETE /identity/v3/projects/2ba67c6da87545f7847cd4afbf0d32cd => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:39.058169 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067976da-63d0-406c-adcb-967372c23853 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:39.058451 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067976da-63d0-406c-adcb-967372c23853 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:39.058744 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067976da-63d0-406c-adcb-967372c23853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:39.078409 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-067976da-63d0-406c-adcb-967372c23853 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:39.079612 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-067976da-63d0-406c-adcb-967372c23853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:39.136494 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1460/2882] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:39.144032 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8608f7-3f1c-4a7d-90af-4c283be5aa5a None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:39.144320 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8608f7-3f1c-4a7d-90af-4c283be5aa5a None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:39.144617 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8608f7-3f1c-4a7d-90af-4c283be5aa5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:39.152779 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:39.165982 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-0f8608f7-3f1c-4a7d-90af-4c283be5aa5a None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:39.167207 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-0f8608f7-3f1c-4a7d-90af-4c283be5aa5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:39.174909 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1423/2883] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:17:39.179527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067976da-63d0-406c-adcb-967372c23853 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:39.189185 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067976da-63d0-406c-adcb-967372c23853 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:39.194262 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f64bf-d0ac-4509-963b-bd01ddff9a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:39.194846 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f64bf-d0ac-4509-963b-bd01ddff9a08 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:39.195162 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f64bf-d0ac-4509-963b-bd01ddff9a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:39.195436 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f64bf-d0ac-4509-963b-bd01ddff9a08 admin admin] PATH_INFO: `/v3/domains` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:39.196084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f64bf-d0ac-4509-963b-bd01ddff9a08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81389) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:39.196369 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f64bf-d0ac-4509-963b-bd01ddff9a08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:39.197955 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f64bf-d0ac-4509-963b-bd01ddff9a08 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:39.209208 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1461/2884] 10.222.0.36 () {58 vars in 1153 bytes} [Fri Sep 2 09:17:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:39.213548 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8608f7-3f1c-4a7d-90af-4c283be5aa5a None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:39.222550 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8608f7-3f1c-4a7d-90af-4c283be5aa5a None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:39.227232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d133c87-d9c9-4649-b33f-4e81568843e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:39.227819 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d133c87-d9c9-4649-b33f-4e81568843e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:39.228120 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d133c87-d9c9-4649-b33f-4e81568843e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:39.228415 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d133c87-d9c9-4649-b33f-4e81568843e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:39.229044 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d133c87-d9c9-4649-b33f-4e81568843e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:39.230906 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d133c87-d9c9-4649-b33f-4e81568843e9 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:39.258259 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1424/2885] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:17:39 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:17:39.262287 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f64bf-d0ac-4509-963b-bd01ddff9a08 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:39.272064 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f64bf-d0ac-4509-963b-bd01ddff9a08 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:39.275580 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:39.328948 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c18598-a3f4-485e-a8a1-fcde0acae396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:39.329537 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c18598-a3f4-485e-a8a1-fcde0acae396 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:39.329812 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c18598-a3f4-485e-a8a1-fcde0acae396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:39.330080 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c18598-a3f4-485e-a8a1-fcde0acae396 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:39.330703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c18598-a3f4-485e-a8a1-fcde0acae396 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:39.332634 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c18598-a3f4-485e-a8a1-fcde0acae396 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:39.365843 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1462/2886] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:17:39 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:17:39.367895 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d133c87-d9c9-4649-b33f-4e81568843e9 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:39.378060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d133c87-d9c9-4649-b33f-4e81568843e9 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:39.383317 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c109400-22a6-4370-a0c9-6feff95ed5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:39.383921 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c109400-22a6-4370-a0c9-6feff95ed5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:39.384188 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c109400-22a6-4370-a0c9-6feff95ed5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:39.384458 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c109400-22a6-4370-a0c9-6feff95ed5f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:39.385100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c109400-22a6-4370-a0c9-6feff95ed5f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:39.386675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c109400-22a6-4370-a0c9-6feff95ed5f0 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:39.396583 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1425/2887] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:17:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:17:39.400609 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c18598-a3f4-485e-a8a1-fcde0acae396 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:39.410490 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c18598-a3f4-485e-a8a1-fcde0acae396 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:39.414869 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ad1ff-d0c2-4c98-a7a0-9e5ae1566dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:39.415491 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ad1ff-d0c2-4c98-a7a0-9e5ae1566dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:39.415767 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ad1ff-d0c2-4c98-a7a0-9e5ae1566dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:39.416040 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ad1ff-d0c2-4c98-a7a0-9e5ae1566dd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:39.416669 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ad1ff-d0c2-4c98-a7a0-9e5ae1566dd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:39.418233 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ad1ff-d0c2-4c98-a7a0-9e5ae1566dd9 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:39.429462 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1463/2888] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:17:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:17:39.433529 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c109400-22a6-4370-a0c9-6feff95ed5f0 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:39.444054 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c109400-22a6-4370-a0c9-6feff95ed5f0 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:39.448757 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e70da0-a8d1-480f-898f-0ea07751d40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:39.449401 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e70da0-a8d1-480f-898f-0ea07751d40b admin admin] REQUEST_METHOD: `PUT` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:39.449678 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e70da0-a8d1-480f-898f-0ea07751d40b admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:39.449950 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e70da0-a8d1-480f-898f-0ea07751d40b admin admin] PATH_INFO: `/v3/projects/b133928ce6ac4acaa1b0c224909c76fd/users/fa92771235f04c668690cd05d4a2b44c/roles/08c2a54a85ef4bb9892a5769ef8a5d95` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:39.464206 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e70da0-a8d1-480f-898f-0ea07751d40b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b133928ce6ac4acaa1b0c224909c76fd, user_id=fa92771235f04c668690cd05d4a2b44c, role_id=08c2a54a85ef4bb9892a5769ef8a5d95)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:39.466823 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e70da0-a8d1-480f-898f-0ea07751d40b admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:39.479355 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1426/2889] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:17:39 2022] PUT /identity/v3/projects/b133928ce6ac4acaa1b0c224909c76fd/users/fa92771235f04c668690cd05d4a2b44c/roles/08c2a54a85ef4bb9892a5769ef8a5d95 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:39.484270 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caed37-e927-4c42-8f43-7d6bb22e59b8 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:39.484561 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caed37-e927-4c42-8f43-7d6bb22e59b8 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:39.484831 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9caed37-e927-4c42-8f43-7d6bb22e59b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:39.506163 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-c9caed37-e927-4c42-8f43-7d6bb22e59b8 None None] MFA Rules not processed for user `95efeb6f9f5f497f9453dc5e9764070a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:39.507392 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-c9caed37-e927-4c42-8f43-7d6bb22e59b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:39.566549 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1464/2890] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:39.576066 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e70da0-a8d1-480f-898f-0ea07751d40b admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:39.577734 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-f6e70da0-a8d1-480f-898f-0ea07751d40b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:39.632887 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e70da0-a8d1-480f-898f-0ea07751d40b admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:39.636529 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf92e9-c7d4-4618-9a9f-e33f81c4fefd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:39.637144 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf92e9-c7d4-4618-9a9f-e33f81c4fefd service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:39.637425 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf92e9-c7d4-4618-9a9f-e33f81c4fefd service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:39.637692 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf92e9-c7d4-4618-9a9f-e33f81c4fefd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:39.646363 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdf92e9-c7d4-4618-9a9f-e33f81c4fefd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:39.647803 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdf92e9-c7d4-4618-9a9f-e33f81c4fefd service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:39.656878 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1427/2891] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:17:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:17:42.779244 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f537eb-0f46-4ae5-8eb6-4b0e7f74c461 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:42.779813 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f537eb-0f46-4ae5-8eb6-4b0e7f74c461 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:42.779813 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f537eb-0f46-4ae5-8eb6-4b0e7f74c461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:42.800010 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-99f537eb-0f46-4ae5-8eb6-4b0e7f74c461 None None] MFA Rules not processed for user `7e1f6e80465a44d0a90200ada3f43e23`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:42.801840 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-99f537eb-0f46-4ae5-8eb6-4b0e7f74c461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:42.867946 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1465/2892] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:43.448856 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf92e9-c7d4-4618-9a9f-e33f81c4fefd service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:43.451663 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-0cdf92e9-c7d4-4618-9a9f-e33f81c4fefd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:43.468738 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf92e9-c7d4-4618-9a9f-e33f81c4fefd service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:43.473615 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b5b02-3a49-4081-b503-96e63eba2579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:43.474251 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b5b02-3a49-4081-b503-96e63eba2579 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:43.474528 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b5b02-3a49-4081-b503-96e63eba2579 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:43.475913 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b5b02-3a49-4081-b503-96e63eba2579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:43.476603 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-303b5b02-3a49-4081-b503-96e63eba2579 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:43.487181 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b5b02-3a49-4081-b503-96e63eba2579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:43.488117 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b5b02-3a49-4081-b503-96e63eba2579 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:43.498321 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1428/2893] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:17:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:17:44.290438 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd292b-b2b3-42b3-a9cd-2644648b776f None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:44.290932 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd292b-b2b3-42b3-a9cd-2644648b776f None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:44.291048 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd292b-b2b3-42b3-a9cd-2644648b776f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:44.328955 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-21bd292b-b2b3-42b3-a9cd-2644648b776f None None] MFA Rules not processed for user `fa92771235f04c668690cd05d4a2b44c`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:44.330164 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-21bd292b-b2b3-42b3-a9cd-2644648b776f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:44.352695 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:44.409845 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1466/2894] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:44.415176 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b5b02-3a49-4081-b503-96e63eba2579 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:44.427303 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b5b02-3a49-4081-b503-96e63eba2579 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:44.432874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe03a32-ae98-4a37-b5c2-a4482df4f979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:44.433743 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe03a32-ae98-4a37-b5c2-a4482df4f979 admin admin] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:44.434181 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe03a32-ae98-4a37-b5c2-a4482df4f979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:44.434608 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe03a32-ae98-4a37-b5c2-a4482df4f979 admin admin] PATH_INFO: `/v3/projects` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:44.435446 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe03a32-ae98-4a37-b5c2-a4482df4f979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:44.437675 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe03a32-ae98-4a37-b5c2-a4482df4f979 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:44.469526 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1429/2895] 10.222.0.36 () {60 vars in 1153 bytes} [Fri Sep 2 09:17:44 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:17:44.474274 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd292b-b2b3-42b3-a9cd-2644648b776f None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:44.485338 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd292b-b2b3-42b3-a9cd-2644648b776f None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:44.549654 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bb641-be67-4ae9-ac87-4598e79ffee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:44.550668 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bb641-be67-4ae9-ac87-4598e79ffee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:44.550947 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bb641-be67-4ae9-ac87-4598e79ffee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:44.551855 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bb641-be67-4ae9-ac87-4598e79ffee4 admin admin] PATH_INFO: `/v3/users` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:44.552516 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bb641-be67-4ae9-ac87-4598e79ffee4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:44.554424 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bb641-be67-4ae9-ac87-4598e79ffee4 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:44.576139 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1467/2896] 10.222.0.36 () {60 vars in 1144 bytes} [Fri Sep 2 09:17:44 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:17:44.579972 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe03a32-ae98-4a37-b5c2-a4482df4f979 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:44.590384 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe03a32-ae98-4a37-b5c2-a4482df4f979 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:44.595117 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766fdcea-1b82-4395-9964-4006256322c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:44.595766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fdcea-1b82-4395-9964-4006256322c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:44.596043 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fdcea-1b82-4395-9964-4006256322c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:44.596310 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fdcea-1b82-4395-9964-4006256322c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:44.596992 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766fdcea-1b82-4395-9964-4006256322c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:44.598776 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766fdcea-1b82-4395-9964-4006256322c3 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:44.609614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1430/2897] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:17:44.614272 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bb641-be67-4ae9-ac87-4598e79ffee4 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:44.624943 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bb641-be67-4ae9-ac87-4598e79ffee4 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:44.630285 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd33a6-a093-4258-bb84-f0ab1200aeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:44.630940 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd33a6-a093-4258-bb84-f0ab1200aeb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:44.631284 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd33a6-a093-4258-bb84-f0ab1200aeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:44.631555 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd33a6-a093-4258-bb84-f0ab1200aeb6 admin admin] PATH_INFO: `/v3/projects/dafeb4337801421ead10afcf3c00b829/users/721bb99123784c5bafb8680e0a07497a/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:44.648587 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd33a6-a093-4258-bb84-f0ab1200aeb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dafeb4337801421ead10afcf3c00b829, user_id=721bb99123784c5bafb8680e0a07497a, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:44.651688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd33a6-a093-4258-bb84-f0ab1200aeb6 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:44.666069 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1468/2898] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:17:44 2022] PUT /identity/v3/projects/dafeb4337801421ead10afcf3c00b829/users/721bb99123784c5bafb8680e0a07497a/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:44.669900 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766fdcea-1b82-4395-9964-4006256322c3 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:44.681505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766fdcea-1b82-4395-9964-4006256322c3 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:44.733142 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:44.741835 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e1fcf-8692-4b7f-859a-59e64eae89d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:44.742442 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e1fcf-8692-4b7f-859a-59e64eae89d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:44.742727 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e1fcf-8692-4b7f-859a-59e64eae89d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:44.742995 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e1fcf-8692-4b7f-859a-59e64eae89d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:44.743679 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e1fcf-8692-4b7f-859a-59e64eae89d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:44.745515 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e1fcf-8692-4b7f-859a-59e64eae89d8 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:44.757614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1431/2899] 10.222.0.36 () {58 vars in 1122 bytes} [Fri Sep 2 09:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:17:44.761586 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd33a6-a093-4258-bb84-f0ab1200aeb6 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:44.769577 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e1fcf-8692-4b7f-859a-59e64eae89d8 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:44.771302 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-627e1fcf-8692-4b7f-859a-59e64eae89d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:44.772073 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd33a6-a093-4258-bb84-f0ab1200aeb6 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:44.776547 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009cca2-1a54-45ca-89a3-f730989dbd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:44.777186 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009cca2-1a54-45ca-89a3-f730989dbd14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:44.777463 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009cca2-1a54-45ca-89a3-f730989dbd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:44.777735 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009cca2-1a54-45ca-89a3-f730989dbd14 admin admin] PATH_INFO: `/v3/projects/dafeb4337801421ead10afcf3c00b829/users/721bb99123784c5bafb8680e0a07497a/roles/a0a76de72fba4036bc1e3e64df717c7f` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:44.779649 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009cca2-1a54-45ca-89a3-f730989dbd14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dafeb4337801421ead10afcf3c00b829, user_id=721bb99123784c5bafb8680e0a07497a, role_id=a0a76de72fba4036bc1e3e64df717c7f)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:44.782252 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009cca2-1a54-45ca-89a3-f730989dbd14 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:44.792786 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1469/2900] 10.222.0.36 () {60 vars in 1483 bytes} [Fri Sep 2 09:17:44 2022] PUT /identity/v3/projects/dafeb4337801421ead10afcf3c00b829/users/721bb99123784c5bafb8680e0a07497a/roles/a0a76de72fba4036bc1e3e64df717c7f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:44.847459 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e1fcf-8692-4b7f-859a-59e64eae89d8 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:44.945863 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56636f-c496-4732-9723-ef9b4fba31d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a0f0fdbbd20492d8c35ec4b73404632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:44.946619 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56636f-c496-4732-9723-ef9b4fba31d2 service placement] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:44.946977 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56636f-c496-4732-9723-ef9b4fba31d2 service placement] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:44.947441 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e56636f-c496-4732-9723-ef9b4fba31d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:44.949411 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-2e56636f-c496-4732-9723-ef9b4fba31d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:44.963958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e56636f-c496-4732-9723-ef9b4fba31d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:44.965812 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e56636f-c496-4732-9723-ef9b4fba31d2 service placement] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:44.983493 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1432/2901] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:17:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:17:50.338626 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f82efe-5912-453c-a953-24bab56ceb68 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:50.339201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f82efe-5912-453c-a953-24bab56ceb68 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:50.339201 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f82efe-5912-453c-a953-24bab56ceb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:50.377764 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-f5f82efe-5912-453c-a953-24bab56ceb68 None None] MFA Rules not processed for user `721bb99123784c5bafb8680e0a07497a`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:50.379774 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f5f82efe-5912-453c-a953-24bab56ceb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:50.481946 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:50.489196 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1470/2902] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:50.501650 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56636f-c496-4732-9723-ef9b4fba31d2 service placement] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:50.503169 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-2e56636f-c496-4732-9723-ef9b4fba31d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:50.578997 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e56636f-c496-4732-9723-ef9b4fba31d2 service placement] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:50.582665 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055a76c-06f4-4fb6-a09a-0d8cbdd04e7c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2474d89f10764afbbc5e740dccaae4df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:50.583316 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055a76c-06f4-4fb6-a09a-0d8cbdd04e7c service glance] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:50.583588 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055a76c-06f4-4fb6-a09a-0d8cbdd04e7c service glance] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:50.583855 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055a76c-06f4-4fb6-a09a-0d8cbdd04e7c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:50.623601 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=100 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:50.643336 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055a76c-06f4-4fb6-a09a-0d8cbdd04e7c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:50.644808 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055a76c-06f4-4fb6-a09a-0d8cbdd04e7c service glance] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:50.654267 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1433/2903] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:17:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:17:50.779590 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca59a1-e418-4e45-9dfc-73a85953e72d None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:50.779884 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca59a1-e418-4e45-9dfc-73a85953e72d None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:50.780182 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca59a1-e418-4e45-9dfc-73a85953e72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:50.813953 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-41ca59a1-e418-4e45-9dfc-73a85953e72d None None] MFA Rules not processed for user `76ebecce283f491e82a961d7f739878f`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:50.815213 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-41ca59a1-e418-4e45-9dfc-73a85953e72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:50.875949 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1471/2904] 10.222.0.36 () {58 vars in 1009 bytes} [Fri Sep 2 09:17:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:17:50.925374 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055a76c-06f4-4fb6-a09a-0d8cbdd04e7c service glance] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:50.927632 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-8055a76c-06f4-4fb6-a09a-0d8cbdd04e7c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:51.005987 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055a76c-06f4-4fb6-a09a-0d8cbdd04e7c service glance] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:51.011046 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c77fa5-3ee4-4180-9dc0-4f0de15dbb0b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92318948dcf1407b9fc8aa5c199787ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:51.011682 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c77fa5-3ee4-4180-9dc0-4f0de15dbb0b service swift] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:51.011958 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c77fa5-3ee4-4180-9dc0-4f0de15dbb0b service swift] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:51.012228 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c77fa5-3ee4-4180-9dc0-4f0de15dbb0b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:51.021443 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c77fa5-3ee4-4180-9dc0-4f0de15dbb0b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:51.023005 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c77fa5-3ee4-4180-9dc0-4f0de15dbb0b service swift] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:51.031172 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1434/2905] 10.222.0.36 () {60 vars in 1384 bytes} [Fri Sep 2 09:17:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:17:59.498166 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c20790-1df1-49fe-b6b1-43a418332f99 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:59.498688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c20790-1df1-49fe-b6b1-43a418332f99 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:59.498803 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c20790-1df1-49fe-b6b1-43a418332f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:59.518994 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-79c20790-1df1-49fe-b6b1-43a418332f99 None None] MFA Rules not processed for user `fa92771235f04c668690cd05d4a2b44c`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:59.520238 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-79c20790-1df1-49fe-b6b1-43a418332f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:59.526959 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1472/2906] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:17:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:18:01.027993 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=42 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:18:01.963424 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c77fa5-3ee4-4180-9dc0-4f0de15dbb0b service swift] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:18:02.030157 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c77fa5-3ee4-4180-9dc0-4f0de15dbb0b service swift] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:18:02.036171 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca96b7-1675-456e-857a-b02100a6ec3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:18:02.036766 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca96b7-1675-456e-857a-b02100a6ec3b service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:18:02.037041 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca96b7-1675-456e-857a-b02100a6ec3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:18:02.037307 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca96b7-1675-456e-857a-b02100a6ec3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:18:02.039060 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-42ca96b7-1675-456e-857a-b02100a6ec3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:18:02.111474 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca96b7-1675-456e-857a-b02100a6ec3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:18:02.112906 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca96b7-1675-456e-857a-b02100a6ec3b service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:18:02.122994 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1435/2907] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:18:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:18:02.280227 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cc509-9dec-4b8c-8c29-d243370dd796 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:18:02.280672 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cc509-9dec-4b8c-8c29-d243370dd796 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:18:02.280771 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537cc509-9dec-4b8c-8c29-d243370dd796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:18:02.307381 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-537cc509-9dec-4b8c-8c29-d243370dd796 None None] MFA Rules not processed for user `76ebecce283f491e82a961d7f739878f`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:18:02.308692 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-537cc509-9dec-4b8c-8c29-d243370dd796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:18:02.316511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1473/2908] 10.222.0.36 () {58 vars in 1009 bytes} [Fri Sep 2 09:18:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:18:02.330245 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca96b7-1675-456e-857a-b02100a6ec3b service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:18:02.343027 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca96b7-1675-456e-857a-b02100a6ec3b service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:18:02.348443 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcdd684-0e76-4c6e-8cc2-e0d01b4d7641 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92318948dcf1407b9fc8aa5c199787ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:18:02.349327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcdd684-0e76-4c6e-8cc2-e0d01b4d7641 service swift] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:18:02.349621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcdd684-0e76-4c6e-8cc2-e0d01b4d7641 service swift] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:18:02.349893 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcdd684-0e76-4c6e-8cc2-e0d01b4d7641 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:18:02.359676 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcdd684-0e76-4c6e-8cc2-e0d01b4d7641 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:18:02.361319 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcdd684-0e76-4c6e-8cc2-e0d01b4d7641 service swift] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:18:02.384504 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1436/2909] 10.222.0.36 () {60 vars in 1384 bytes} [Fri Sep 2 09:18:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:18:12.300883 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=36 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:18:12.366614 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=56 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:18:28.011159 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537cc509-9dec-4b8c-8c29-d243370dd796 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:18:28.020582 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537cc509-9dec-4b8c-8c29-d243370dd796 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:18:28.024123 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4fb361-5f05-456f-9704-49312ab314ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:18:28.024713 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fb361-5f05-456f-9704-49312ab314ea service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:18:28.024989 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fb361-5f05-456f-9704-49312ab314ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:18:28.025256 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fb361-5f05-456f-9704-49312ab314ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:18:28.033927 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4fb361-5f05-456f-9704-49312ab314ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:18:28.035375 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4fb361-5f05-456f-9704-49312ab314ea service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:18:28.044532 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1474/2910] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:18:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:18:38.040505 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=6 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:18:54.087276 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcdd684-0e76-4c6e-8cc2-e0d01b4d7641 service swift] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:18:54.096943 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcdd684-0e76-4c6e-8cc2-e0d01b4d7641 service swift] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:18:54.100717 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b642bb5-4374-4a44-b560-211adf7b6046 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:18:54.101309 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b642bb5-4374-4a44-b560-211adf7b6046 service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:18:54.102226 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b642bb5-4374-4a44-b560-211adf7b6046 service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:18:54.102496 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b642bb5-4374-4a44-b560-211adf7b6046 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:18:54.104261 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-2b642bb5-4374-4a44-b560-211adf7b6046 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:18:54.174796 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b642bb5-4374-4a44-b560-211adf7b6046 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:18:54.176278 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b642bb5-4374-4a44-b560-211adf7b6046 service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:18:54.187403 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1437/2911] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:18:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:18:55.472368 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4fb361-5f05-456f-9704-49312ab314ea service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:18:55.481502 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4fb361-5f05-456f-9704-49312ab314ea service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:18:55.485073 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8b6a7-827c-4353-b881-0baaee768082 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:18:55.485655 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8b6a7-827c-4353-b881-0baaee768082 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:18:55.485929 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8b6a7-827c-4353-b881-0baaee768082 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:18:55.486197 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8b6a7-827c-4353-b881-0baaee768082 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:18:55.487933 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-8aa8b6a7-827c-4353-b881-0baaee768082 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:18:55.501473 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8b6a7-827c-4353-b881-0baaee768082 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:18:55.502884 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8b6a7-827c-4353-b881-0baaee768082 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:18:55.513599 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1475/2912] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:18:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:19:04.182287 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=34 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:19:05.508804 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=6 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:19:43.547481 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b642bb5-4374-4a44-b560-211adf7b6046 service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:19:43.549118 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-2b642bb5-4374-4a44-b560-211adf7b6046 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:19:43.568492 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b642bb5-4374-4a44-b560-211adf7b6046 service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:19:43.577993 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71420a7-7ce0-48b5-bcd7-762655d0c95b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:19:43.578828 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71420a7-7ce0-48b5-bcd7-762655d0c95b service nova] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:19:43.579232 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71420a7-7ce0-48b5-bcd7-762655d0c95b service nova] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:19:43.579606 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71420a7-7ce0-48b5-bcd7-762655d0c95b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:19:43.581815 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-d71420a7-7ce0-48b5-bcd7-762655d0c95b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:19:43.597546 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71420a7-7ce0-48b5-bcd7-762655d0c95b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:19:43.599952 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71420a7-7ce0-48b5-bcd7-762655d0c95b service nova] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:19:43.612092 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1438/2913] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:19:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:19:43.704695 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8b6a7-827c-4353-b881-0baaee768082 service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:19:43.714698 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8b6a7-827c-4353-b881-0baaee768082 service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:19:43.718723 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51cdf4-bd0a-44d9-b3a9-754118508406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:19:43.719374 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51cdf4-bd0a-44d9-b3a9-754118508406 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:19:43.719839 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51cdf4-bd0a-44d9-b3a9-754118508406 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:19:43.720273 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51cdf4-bd0a-44d9-b3a9-754118508406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:19:43.722509 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-ff51cdf4-bd0a-44d9-b3a9-754118508406 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:19:43.735208 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff51cdf4-bd0a-44d9-b3a9-754118508406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:19:43.737987 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff51cdf4-bd0a-44d9-b3a9-754118508406 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:19:43.748762 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1476/2914] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:19:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:19:51.491804 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234b2cd-3347-4d5c-ad3d-ba4ff00bfaf3 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:19:51.492271 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234b2cd-3347-4d5c-ad3d-ba4ff00bfaf3 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:19:51.492375 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1234b2cd-3347-4d5c-ad3d-ba4ff00bfaf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:19:51.539760 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-1234b2cd-3347-4d5c-ad3d-ba4ff00bfaf3 None None] MFA Rules not processed for user `76ebecce283f491e82a961d7f739878f`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:19:51.540950 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1234b2cd-3347-4d5c-ad3d-ba4ff00bfaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:19:51.557371 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1439/2915] 10.222.0.36 () {58 vars in 967 bytes} [Fri Sep 2 09:19:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:19:51.597258 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51cdf4-bd0a-44d9-b3a9-754118508406 service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:19:51.619208 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51cdf4-bd0a-44d9-b3a9-754118508406 service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:19:51.624794 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9028d98-5c89-4c27-9555-776501dee7af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '92318948dcf1407b9fc8aa5c199787ae', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:19:51.625377 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9028d98-5c89-4c27-9555-776501dee7af service swift] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:19:51.625652 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9028d98-5c89-4c27-9555-776501dee7af service swift] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:19:51.625922 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9028d98-5c89-4c27-9555-776501dee7af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:19:51.633780 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9028d98-5c89-4c27-9555-776501dee7af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:19:51.635361 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9028d98-5c89-4c27-9555-776501dee7af service swift] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:19:51.643170 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1477/2916] 10.222.0.36 () {60 vars in 1384 bytes} [Fri Sep 2 09:19:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:19:58.060008 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234b2cd-3347-4d5c-ad3d-ba4ff00bfaf3 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:19:58.061624 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-1234b2cd-3347-4d5c-ad3d-ba4ff00bfaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:19:58.078573 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1234b2cd-3347-4d5c-ad3d-ba4ff00bfaf3 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:19:58.083788 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab99fcb-d1b2-4f32-ab86-c32450d68e64 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a0f0fdbbd20492d8c35ec4b73404632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:19:58.084395 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab99fcb-d1b2-4f32-ab86-c32450d68e64 service placement] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:19:58.084863 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab99fcb-d1b2-4f32-ab86-c32450d68e64 service placement] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:19:58.085336 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab99fcb-d1b2-4f32-ab86-c32450d68e64 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:19:58.087058 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-4ab99fcb-d1b2-4f32-ab86-c32450d68e64 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:19:58.100814 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab99fcb-d1b2-4f32-ab86-c32450d68e64 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:19:58.102801 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab99fcb-d1b2-4f32-ab86-c32450d68e64 service placement] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:19:58.119605 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1440/2917] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:19:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:19:58.137067 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9028d98-5c89-4c27-9555-776501dee7af service swift] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:19:58.207892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9028d98-5c89-4c27-9555-776501dee7af service swift] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:19:58.212319 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2181b-cb48-455a-b008-30f7418743c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:19:58.213193 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2181b-cb48-455a-b008-30f7418743c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:19:58.213466 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2181b-cb48-455a-b008-30f7418743c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:19:58.213730 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d2181b-cb48-455a-b008-30f7418743c3 admin admin] PATH_INFO: `/v3/users/fa92771235f04c668690cd05d4a2b44c` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:19:58.214823 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d2181b-cb48-455a-b008-30f7418743c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa92771235f04c668690cd05d4a2b44c)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:19:58.216785 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d2181b-cb48-455a-b008-30f7418743c3 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:19:58.263055 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f5d2181b-cb48-455a-b008-30f7418743c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa92771235f04c668690cd05d4a2b44c'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:19:58.269847 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f5d2181b-cb48-455a-b008-30f7418743c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa92771235f04c668690cd05d4a2b44c'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:19:58.277071 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f5d2181b-cb48-455a-b008-30f7418743c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa92771235f04c668690cd05d4a2b44c'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:19:58.277783 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f5d2181b-cb48-455a-b008-30f7418743c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa92771235f04c668690cd05d4a2b44c'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:19:58.278576 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-f5d2181b-cb48-455a-b008-30f7418743c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa92771235f04c668690cd05d4a2b44c'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:19:58.295908 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1478/2918] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:19:58 2022] DELETE /identity/v3/users/fa92771235f04c668690cd05d4a2b44c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:19:58.465402 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab99fcb-d1b2-4f32-ab86-c32450d68e64 service placement] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:19:58.476860 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab99fcb-d1b2-4f32-ab86-c32450d68e64 service placement] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:19:58.533748 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386028b1-1414-487c-8e9c-1ec074ab02d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:19:58.534372 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386028b1-1414-487c-8e9c-1ec074ab02d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:19:58.534655 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386028b1-1414-487c-8e9c-1ec074ab02d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:19:58.535059 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386028b1-1414-487c-8e9c-1ec074ab02d4 admin admin] PATH_INFO: `/v3/projects/b133928ce6ac4acaa1b0c224909c76fd` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:19:58.536207 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386028b1-1414-487c-8e9c-1ec074ab02d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b133928ce6ac4acaa1b0c224909c76fd)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:19:58.538099 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386028b1-1414-487c-8e9c-1ec074ab02d4 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:19:58.582132 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-386028b1-1414-487c-8e9c-1ec074ab02d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b133928ce6ac4acaa1b0c224909c76fd', 'request_id': 'req-386028b1-1414-487c-8e9c-1ec074ab02d4'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:19:58.588292 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-386028b1-1414-487c-8e9c-1ec074ab02d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b133928ce6ac4acaa1b0c224909c76fd', 'request_id': 'req-386028b1-1414-487c-8e9c-1ec074ab02d4'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:19:58.593700 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-386028b1-1414-487c-8e9c-1ec074ab02d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b133928ce6ac4acaa1b0c224909c76fd', 'request_id': 'req-386028b1-1414-487c-8e9c-1ec074ab02d4'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:19:58.605156 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-386028b1-1414-487c-8e9c-1ec074ab02d4 admin admin] The token cache is being invalidate because project b133928ce6ac4acaa1b0c224909c76fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:19:58.605444 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-386028b1-1414-487c-8e9c-1ec074ab02d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:19:58.606058 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-386028b1-1414-487c-8e9c-1ec074ab02d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:19:58.607681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1441/2919] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:19:58 2022] DELETE /identity/v3/projects/b133928ce6ac4acaa1b0c224909c76fd => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:19:58.611613 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2181b-cb48-455a-b008-30f7418743c3 admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:19:58.612988 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-f5d2181b-cb48-455a-b008-30f7418743c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:19:58.626079 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d2181b-cb48-455a-b008-30f7418743c3 admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:19:58.666617 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:19:58.682340 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09c52a-9a21-4cde-9922-a2c316a013e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:19:58.682964 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09c52a-9a21-4cde-9922-a2c316a013e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:19:58.683276 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09c52a-9a21-4cde-9922-a2c316a013e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:19:58.683579 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09c52a-9a21-4cde-9922-a2c316a013e6 admin admin] PATH_INFO: `/v3/users/721bb99123784c5bafb8680e0a07497a` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:19:58.685282 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09c52a-9a21-4cde-9922-a2c316a013e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=721bb99123784c5bafb8680e0a07497a)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:19:58.687308 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09c52a-9a21-4cde-9922-a2c316a013e6 admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:19:58.731474 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-2d09c52a-9a21-4cde-9922-a2c316a013e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '721bb99123784c5bafb8680e0a07497a'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:19:58.738202 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-2d09c52a-9a21-4cde-9922-a2c316a013e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '721bb99123784c5bafb8680e0a07497a'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:19:58.745384 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-2d09c52a-9a21-4cde-9922-a2c316a013e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '721bb99123784c5bafb8680e0a07497a'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:19:58.745993 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-2d09c52a-9a21-4cde-9922-a2c316a013e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '721bb99123784c5bafb8680e0a07497a'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:19:58.746620 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-2d09c52a-9a21-4cde-9922-a2c316a013e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '721bb99123784c5bafb8680e0a07497a'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:19:58.764740 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1479/2920] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:19:58 2022] DELETE /identity/v3/users/721bb99123784c5bafb8680e0a07497a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:19:58.932401 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386028b1-1414-487c-8e9c-1ec074ab02d4 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:19:58.933952 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-386028b1-1414-487c-8e9c-1ec074ab02d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:19:58.947726 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386028b1-1414-487c-8e9c-1ec074ab02d4 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:19:58.971637 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:19:59.016838 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d1d1c-7fba-4999-8218-b9c26fbcbbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:19:59.017479 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d1d1c-7fba-4999-8218-b9c26fbcbbe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:19:59.017762 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d1d1c-7fba-4999-8218-b9c26fbcbbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:19:59.018037 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d1d1c-7fba-4999-8218-b9c26fbcbbe2 admin admin] PATH_INFO: `/v3/projects/dafeb4337801421ead10afcf3c00b829` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:19:59.019212 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d1d1c-7fba-4999-8218-b9c26fbcbbe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dafeb4337801421ead10afcf3c00b829)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:19:59.021128 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d1d1c-7fba-4999-8218-b9c26fbcbbe2 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:19:59.070019 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a07d1d1c-7fba-4999-8218-b9c26fbcbbe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dafeb4337801421ead10afcf3c00b829', 'request_id': 'req-a07d1d1c-7fba-4999-8218-b9c26fbcbbe2'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:19:59.077424 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a07d1d1c-7fba-4999-8218-b9c26fbcbbe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dafeb4337801421ead10afcf3c00b829', 'request_id': 'req-a07d1d1c-7fba-4999-8218-b9c26fbcbbe2'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:19:59.082952 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a07d1d1c-7fba-4999-8218-b9c26fbcbbe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dafeb4337801421ead10afcf3c00b829', 'request_id': 'req-a07d1d1c-7fba-4999-8218-b9c26fbcbbe2'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:19:59.096026 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a07d1d1c-7fba-4999-8218-b9c26fbcbbe2 admin admin] The token cache is being invalidate because project dafeb4337801421ead10afcf3c00b829 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81390) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:19:59.096315 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a07d1d1c-7fba-4999-8218-b9c26fbcbbe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:19:59.096939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-a07d1d1c-7fba-4999-8218-b9c26fbcbbe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:19:59.098437 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1442/2921] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:19:58 2022] DELETE /identity/v3/projects/dafeb4337801421ead10afcf3c00b829 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:20:05.731993 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d071612-df3d-4b64-865f-6c3d553118e4 None None] REQUEST_METHOD: `POST` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:20:05.732457 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d071612-df3d-4b64-865f-6c3d553118e4 None None] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:20:05.732567 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d071612-df3d-4b64-865f-6c3d553118e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:20:05.751084 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.auth.core [None req-9d071612-df3d-4b64-865f-6c3d553118e4 None None] MFA Rules not processed for user `7e1f6e80465a44d0a90200ada3f43e23`. Rule list: `[]` (Enabled: `True`). {{(pid=81389) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:20:05.752286 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-9d071612-df3d-4b64-865f-6c3d553118e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:20:05.804919 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1480/2922] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:20:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:20:06.546434 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d1d1c-7fba-4999-8218-b9c26fbcbbe2 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:20:06.547939 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-a07d1d1c-7fba-4999-8218-b9c26fbcbbe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:20:06.617681 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d1d1c-7fba-4999-8218-b9c26fbcbbe2 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:20:06.623250 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321dea4-b2bd-448d-a458-af8d7490e55f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a0f0fdbbd20492d8c35ec4b73404632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:20:06.623852 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321dea4-b2bd-448d-a458-af8d7490e55f service placement] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:20:06.624133 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321dea4-b2bd-448d-a458-af8d7490e55f service placement] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:20:06.624401 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321dea4-b2bd-448d-a458-af8d7490e55f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:20:06.626109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-0321dea4-b2bd-448d-a458-af8d7490e55f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:20:06.637109 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321dea4-b2bd-448d-a458-af8d7490e55f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:20:06.638697 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321dea4-b2bd-448d-a458-af8d7490e55f service placement] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:20:06.648399 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1443/2923] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:20:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:20:15.798562 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=58,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:20:16.644184 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:20:18.387603 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d071612-df3d-4b64-865f-6c3d553118e4 None None] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:20:18.389185 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-9d071612-df3d-4b64-865f-6c3d553118e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:20:18.473126 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d071612-df3d-4b64-865f-6c3d553118e4 None None] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:20:18.479817 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8258e7-c3d6-49d1-8277-7407eddc5d52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:20:18.480377 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8258e7-c3d6-49d1-8277-7407eddc5d52 service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:20:18.480703 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8258e7-c3d6-49d1-8277-7407eddc5d52 service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:20:18.480951 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8258e7-c3d6-49d1-8277-7407eddc5d52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:20:18.482682 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-9b8258e7-c3d6-49d1-8277-7407eddc5d52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:20:18.495520 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8258e7-c3d6-49d1-8277-7407eddc5d52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:20:18.497122 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8258e7-c3d6-49d1-8277-7407eddc5d52 service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:20:18.508277 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1481/2924] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:20:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:20:28.503126 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=34 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:22:09.497237 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2971b87-a44b-4b66-b69b-75299cbe7ce9 None None] REQUEST_METHOD: `POST` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:22:09.497784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2971b87-a44b-4b66-b69b-75299cbe7ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:22:09.497784 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2971b87-a44b-4b66-b69b-75299cbe7ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:22:09.516407 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.auth.core [None req-e2971b87-a44b-4b66-b69b-75299cbe7ce9 None None] MFA Rules not processed for user `7e1f6e80465a44d0a90200ada3f43e23`. Rule list: `[]` (Enabled: `True`). {{(pid=81390) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:22:09.517579 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-e2971b87-a44b-4b66-b69b-75299cbe7ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:22:09.524267 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1444/2925] 10.222.0.36 () {58 vars in 958 bytes} [Fri Sep 2 09:22:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:22:19.510621 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=3 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:22:23.085839 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8258e7-c3d6-49d1-8277-7407eddc5d52 service nova] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:22:23.097385 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8258e7-c3d6-49d1-8277-7407eddc5d52 service nova] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:22:23.104249 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6732645b-bf47-4497-aef5-1b6c3aab17a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cce502a83f5343558c2136c0193e7d85', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:22:23.104974 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732645b-bf47-4497-aef5-1b6c3aab17a7 service nova] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:22:23.105257 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732645b-bf47-4497-aef5-1b6c3aab17a7 service nova] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:22:23.106171 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732645b-bf47-4497-aef5-1b6c3aab17a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:22:23.108259 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6732645b-bf47-4497-aef5-1b6c3aab17a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:22:23.119994 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6732645b-bf47-4497-aef5-1b6c3aab17a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:22:23.123061 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6732645b-bf47-4497-aef5-1b6c3aab17a7 service nova] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:22:23.133018 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1482/2926] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:22:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:22:33.128880 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=6 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:22:51.300899 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2971b87-a44b-4b66-b69b-75299cbe7ce9 None None] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:22:51.302320 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-e2971b87-a44b-4b66-b69b-75299cbe7ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:22:51.317875 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2971b87-a44b-4b66-b69b-75299cbe7ce9 None None] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:22:51.322843 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345a578-4343-4ff4-b636-cfb5c6de0164 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a0f0fdbbd20492d8c35ec4b73404632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:22:51.323475 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345a578-4343-4ff4-b636-cfb5c6de0164 service placement] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:22:51.323755 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345a578-4343-4ff4-b636-cfb5c6de0164 service placement] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:22:51.324026 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345a578-4343-4ff4-b636-cfb5c6de0164 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:22:51.325715 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-4345a578-4343-4ff4-b636-cfb5c6de0164 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:22:51.337429 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345a578-4343-4ff4-b636-cfb5c6de0164 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:22:51.338985 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345a578-4343-4ff4-b636-cfb5c6de0164 service placement] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:22:51.349031 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1445/2927] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:22:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:23:01.344777 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=6 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:23:56.510807 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6732645b-bf47-4497-aef5-1b6c3aab17a7 service nova] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:23:56.512248 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-6732645b-bf47-4497-aef5-1b6c3aab17a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:23:56.578476 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6732645b-bf47-4497-aef5-1b6c3aab17a7 service nova] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:23:56.582251 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd6b0f-3e14-4c6c-b250-8ce11a7bd456 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:23:56.583102 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd6b0f-3e14-4c6c-b250-8ce11a7bd456 service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:23:56.583266 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd6b0f-3e14-4c6c-b250-8ce11a7bd456 service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:23:56.583423 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd6b0f-3e14-4c6c-b250-8ce11a7bd456 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:23:56.585256 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-a9cd6b0f-3e14-4c6c-b250-8ce11a7bd456 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:23:56.600077 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd6b0f-3e14-4c6c-b250-8ce11a7bd456 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:23:56.601521 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd6b0f-3e14-4c6c-b250-8ce11a7bd456 service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:23:56.613700 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1483/2928] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:23:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:23:57.799665 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345a578-4343-4ff4-b636-cfb5c6de0164 service placement] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:23:57.808979 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345a578-4343-4ff4-b636-cfb5c6de0164 service placement] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:23:57.813332 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b12575-f7eb-41d6-8974-e717c903f1eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:23:57.814050 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b12575-f7eb-41d6-8974-e717c903f1eb service neutron] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:23:57.814337 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b12575-f7eb-41d6-8974-e717c903f1eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:23:57.814609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b12575-f7eb-41d6-8974-e717c903f1eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:23:57.816345 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-80b12575-f7eb-41d6-8974-e717c903f1eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:23:57.830554 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b12575-f7eb-41d6-8974-e717c903f1eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:23:57.831935 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b12575-f7eb-41d6-8974-e717c903f1eb service neutron] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:23:57.843474 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1446/2929] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:23:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:24:06.607956 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=26 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:07.838505 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=6 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:08.726417 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd6b0f-3e14-4c6c-b250-8ce11a7bd456 service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:24:08.735874 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd6b0f-3e14-4c6c-b250-8ce11a7bd456 service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:24:08.739471 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998546cb-1954-4c7d-920f-dcd4185ed4bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07a825fddbc04f918bcd631976038cf9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:24:08.740058 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998546cb-1954-4c7d-920f-dcd4185ed4bb service neutron] REQUEST_METHOD: `GET` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:08.740355 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998546cb-1954-4c7d-920f-dcd4185ed4bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:08.740625 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998546cb-1954-4c7d-920f-dcd4185ed4bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:08.742345 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-998546cb-1954-4c7d-920f-dcd4185ed4bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:08.803688 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998546cb-1954-4c7d-920f-dcd4185ed4bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:24:08.805112 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998546cb-1954-4c7d-920f-dcd4185ed4bb service neutron] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:24:08.814527 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1484/2930] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:24:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:24:14.930342 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b12575-f7eb-41d6-8974-e717c903f1eb service neutron] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:24:14.931886 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-80b12575-f7eb-41d6-8974-e717c903f1eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:14.946152 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b12575-f7eb-41d6-8974-e717c903f1eb service neutron] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:24:14.950656 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c38ac-3f67-4e58-9ba4-6cea2fcd05c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:24:14.951307 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c38ac-3f67-4e58-9ba4-6cea2fcd05c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:14.951609 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c38ac-3f67-4e58-9ba4-6cea2fcd05c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:14.951874 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9c38ac-3f67-4e58-9ba4-6cea2fcd05c6 admin admin] PATH_INFO: `/v3/users/7e1f6e80465a44d0a90200ada3f43e23` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:14.952972 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c38ac-3f67-4e58-9ba4-6cea2fcd05c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e1f6e80465a44d0a90200ada3f43e23)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:24:14.954885 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9c38ac-3f67-4e58-9ba4-6cea2fcd05c6 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:24:15.002829 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-df9c38ac-3f67-4e58-9ba4-6cea2fcd05c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e1f6e80465a44d0a90200ada3f43e23'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:24:15.003497 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-df9c38ac-3f67-4e58-9ba4-6cea2fcd05c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e1f6e80465a44d0a90200ada3f43e23'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:24:15.014981 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-df9c38ac-3f67-4e58-9ba4-6cea2fcd05c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e1f6e80465a44d0a90200ada3f43e23'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:24:15.021718 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-df9c38ac-3f67-4e58-9ba4-6cea2fcd05c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e1f6e80465a44d0a90200ada3f43e23'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:24:15.022367 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-df9c38ac-3f67-4e58-9ba4-6cea2fcd05c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e1f6e80465a44d0a90200ada3f43e23'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:24:15.035013 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1447/2931] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:24:14 2022] DELETE /identity/v3/users/7e1f6e80465a44d0a90200ada3f43e23 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:24:15.207995 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998546cb-1954-4c7d-920f-dcd4185ed4bb service neutron] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:24:15.221027 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998546cb-1954-4c7d-920f-dcd4185ed4bb service neutron] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:24:15.279678 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0f8e0-3bae-41e5-a893-3344d7989e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:24:15.280925 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0f8e0-3bae-41e5-a893-3344d7989e5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:15.281229 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0f8e0-3bae-41e5-a893-3344d7989e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:15.281531 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0f8e0-3bae-41e5-a893-3344d7989e5b admin admin] PATH_INFO: `/v3/projects/a09b3fcd4fdc424086b7ca6b15d0a4e9` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:15.282676 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0f8e0-3bae-41e5-a893-3344d7989e5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a09b3fcd4fdc424086b7ca6b15d0a4e9)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:24:15.284633 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0f8e0-3bae-41e5-a893-3344d7989e5b admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:24:15.333043 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-7df0f8e0-3bae-41e5-a893-3344d7989e5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a09b3fcd4fdc424086b7ca6b15d0a4e9', 'request_id': 'req-7df0f8e0-3bae-41e5-a893-3344d7989e5b'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:24:15.343705 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-7df0f8e0-3bae-41e5-a893-3344d7989e5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a09b3fcd4fdc424086b7ca6b15d0a4e9', 'request_id': 'req-7df0f8e0-3bae-41e5-a893-3344d7989e5b'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:24:15.351918 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-7df0f8e0-3bae-41e5-a893-3344d7989e5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a09b3fcd4fdc424086b7ca6b15d0a4e9', 'request_id': 'req-7df0f8e0-3bae-41e5-a893-3344d7989e5b'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:24:15.365407 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-7df0f8e0-3bae-41e5-a893-3344d7989e5b admin admin] The token cache is being invalidate because project a09b3fcd4fdc424086b7ca6b15d0a4e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:24:15.365698 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-7df0f8e0-3bae-41e5-a893-3344d7989e5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:24:15.366363 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-7df0f8e0-3bae-41e5-a893-3344d7989e5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:24:15.367892 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1485/2932] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:24:15 2022] DELETE /identity/v3/projects/a09b3fcd4fdc424086b7ca6b15d0a4e9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:24:15.372021 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c38ac-3f67-4e58-9ba4-6cea2fcd05c6 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:24:15.373452 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-df9c38ac-3f67-4e58-9ba4-6cea2fcd05c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:15.386518 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9c38ac-3f67-4e58-9ba4-6cea2fcd05c6 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:24:15.445413 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6de5fa-783c-4dc0-9964-f084ddc18de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:24:15.446049 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6de5fa-783c-4dc0-9964-f084ddc18de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:15.446327 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6de5fa-783c-4dc0-9964-f084ddc18de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:15.446596 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6de5fa-783c-4dc0-9964-f084ddc18de1 admin admin] PATH_INFO: `/v3/users/2b62d2360a2a4e278b45d1a5c45e0cd9` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:15.447758 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6de5fa-783c-4dc0-9964-f084ddc18de1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b62d2360a2a4e278b45d1a5c45e0cd9)` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:24:15.449658 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6de5fa-783c-4dc0-9964-f084ddc18de1 admin admin] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:24:15.501107 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-df6de5fa-783c-4dc0-9964-f084ddc18de1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b62d2360a2a4e278b45d1a5c45e0cd9'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:24:15.501670 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-df6de5fa-783c-4dc0-9964-f084ddc18de1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b62d2360a2a4e278b45d1a5c45e0cd9'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:24:15.513554 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-df6de5fa-783c-4dc0-9964-f084ddc18de1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b62d2360a2a4e278b45d1a5c45e0cd9'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:24:15.520763 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-df6de5fa-783c-4dc0-9964-f084ddc18de1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b62d2360a2a4e278b45d1a5c45e0cd9'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:24:15.521416 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.notifications [None req-df6de5fa-783c-4dc0-9964-f084ddc18de1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b62d2360a2a4e278b45d1a5c45e0cd9'} {{(pid=81390) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:24:15.523045 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:15.535458 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1448/2933] 10.222.0.36 () {58 vars in 1224 bytes} [Fri Sep 2 09:24:15 2022] DELETE /identity/v3/users/2b62d2360a2a4e278b45d1a5c45e0cd9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:24:15.707057 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0f8e0-3bae-41e5-a893-3344d7989e5b admin admin] Authenticating user token {{(pid=81389) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:24:15.708894 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.fernet_utils [None req-7df0f8e0-3bae-41e5-a893-3344d7989e5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81389) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:15.723332 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0f8e0-3bae-41e5-a893-3344d7989e5b admin admin] Validating token access rules against request {{(pid=81389) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:24:15.743842 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:15.782882 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b88c74-0067-4be8-a4c1-e4807cbcea6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95efeb6f9f5f497f9453dc5e9764070a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '32923700004a4c11b8926f21bae6d460', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81389) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:24:15.783459 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b88c74-0067-4be8-a4c1-e4807cbcea6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:15.783741 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b88c74-0067-4be8-a4c1-e4807cbcea6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:15.784015 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b88c74-0067-4be8-a4c1-e4807cbcea6a admin admin] PATH_INFO: `/v3/projects/7d739cfc2558439ea68522c1ecc46287` {{(pid=81389) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:15.785145 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b88c74-0067-4be8-a4c1-e4807cbcea6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d739cfc2558439ea68522c1ecc46287)` {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:24:15.787087 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b88c74-0067-4be8-a4c1-e4807cbcea6a admin admin] RBAC: Authorization granted {{(pid=81389) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:24:15.831810 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-47b88c74-0067-4be8-a4c1-e4807cbcea6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d739cfc2558439ea68522c1ecc46287', 'request_id': 'req-47b88c74-0067-4be8-a4c1-e4807cbcea6a'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:24:15.838931 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-47b88c74-0067-4be8-a4c1-e4807cbcea6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d739cfc2558439ea68522c1ecc46287', 'request_id': 'req-47b88c74-0067-4be8-a4c1-e4807cbcea6a'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:24:15.844511 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-47b88c74-0067-4be8-a4c1-e4807cbcea6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d739cfc2558439ea68522c1ecc46287', 'request_id': 'req-47b88c74-0067-4be8-a4c1-e4807cbcea6a'} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:24:15.855046 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-47b88c74-0067-4be8-a4c1-e4807cbcea6a admin admin] The token cache is being invalidate because project 7d739cfc2558439ea68522c1ecc46287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81389) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:24:15.855354 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-47b88c74-0067-4be8-a4c1-e4807cbcea6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:24:15.856006 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG keystone.notifications [None req-47b88c74-0067-4be8-a4c1-e4807cbcea6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81389) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:24:15.857451 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: [pid: 81389|app: 0|req: 1486/2934] 10.222.0.36 () {58 vars in 1233 bytes} [Fri Sep 2 09:24:15 2022] DELETE /identity/v3/projects/7d739cfc2558439ea68522c1ecc46287 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:24:25.532469 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:25.852263 dv-cinder-855359-2-iscsi devstack@keystone.service[81389]: DEBUG dbcounter [-] [81389] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81389) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:27:57.277791 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6de5fa-783c-4dc0-9964-f084ddc18de1 admin admin] Authenticating user token {{(pid=81390) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:27:57.279393 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-df6de5fa-783c-4dc0-9964-f084ddc18de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:57.414537 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6de5fa-783c-4dc0-9964-f084ddc18de1 admin admin] Validating token access rules against request {{(pid=81390) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:27:57.476884 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844f5692-d0ef-4bb1-9ec0-ccab770101b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a0f0fdbbd20492d8c35ec4b73404632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eef80493db09437b88f2b147a59928c1', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81390) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:27:57.478390 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f5692-d0ef-4bb1-9ec0-ccab770101b3 service placement] REQUEST_METHOD: `GET` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:57.478849 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f5692-d0ef-4bb1-9ec0-ccab770101b3 service placement] SCRIPT_NAME: `/identity` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:57.479149 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844f5692-d0ef-4bb1-9ec0-ccab770101b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81390) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:57.481758 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.fernet_utils [None req-844f5692-d0ef-4bb1-9ec0-ccab770101b3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81390) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:57.493369 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f5692-d0ef-4bb1-9ec0-ccab770101b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:27:57.495100 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844f5692-d0ef-4bb1-9ec0-ccab770101b3 service placement] RBAC: Authorization granted {{(pid=81390) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:27:57.506368 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: [pid: 81390|app: 0|req: 1449/2935] 10.222.0.36 () {60 vars in 1365 bytes} [Fri Sep 2 09:27:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:28:07.501810 dv-cinder-855359-2-iscsi devstack@keystone.service[81390]: DEBUG dbcounter [-] [81390] Writing DB stats keystone:SELECT=89 {{(pid=81390) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}